


Institutional Archive of the Naval Postgraduate School 


Calhoun: The NPS Institutional Archive 
DSpace Repository 


Theses and Dissertations 1. Thesis and Dissertation Collection, all items 


1974-06 


The personnel assessment center: a review 
and suggested application 


Allen, Galen Bruce. 


Monterey, California. Naval Postgraduate School 


http://hdl.handle.net/10945/17170 


Downloaded from NPS Archive: Calhoun 


| Calhoun is the Naval Postgraduate School's public access digital repository for 
D U DLEY research materials and institutional publications created by the NPS community. 
qe Calhoun is named for Professor of Mathematics Guy K. Calhoun, NP3's first 
KNOX appointed — and published — scholarly author. 


LIBRARY Dudley Knox Library / Naval Postgraduate School 
411 Dyer Road / 1 University Circle 
Monterey, California USA 93943 





http://www.nps.edu/library 











ae 


NAVAL POSTGRADUATE SCH 


Monterey, Galifornia 








THESIS 
THE PERSONNEL ASSESSMENT CENTER: 
A REVIEW AND SUGGESTED APPLICATION 


by 


Galen Bruce Allen 


June 1974 





Thesis Advisor: R. S. Elster 


Approved for public release; distribution unlimited. 


T 62-495 





SECURITY CLASSIFICATION OF THIS PAGE (When Dete Entered) 


REPORT DOCUMENTATION PAGE 


t. REPORT NUMBER 


4. TITLE (and Subtitle) 


The Personnel Assessment Center: 
A Review and Suggested Application 






READ INSTRUCTIONS 
BEFORE COMPLETING FORM 
2. GOVT ACCESSION NO. 3. RECIPIENT'S CATALOG NUMGER 














- TYPE OF REPORT & PERIOO COVERED 
Master's Thesis; 
June 1974 


6. PERFORMING ORG. REPORT NUMBER 


8. CONTRACT OR GRANT NUMIER(a) 








7. AUTHOR(e) 


Galen Bruce Allen 







10. PROGRAM ELEMENT, PROJECT, TASK 
AREA & WORK UNIT NUMBERS 


12. REPORT OATE 
June 1974 
13. NUMBER OF PAGES 


MOS 


PIS. SECURITY CLASS. 


9. PERFORMING ORGANIZATION NAME AND ADDRESS 









Naval Postgraduate School 
Monterey, “California “93940 










11. CONTROLLING OFFICE NAME AND ADDRESS 







Naval Postgraduate School 
Monterey, California 93940 


14. MONITORING AGENCY NAME & ADDRESS(I! different trom Controlling Oltice) 


















(of thie report) 






Unclassified 


1Se, CECLASSIFICATION/ DOWNGRADING 
SCHEDULE 






Naval Postgraduate School 
Monterey, California 93940 





16. DISTRIBUTION STATEMENT (of thie Report) 


Approved for public release; distribution unlimited. 


4 


17. DISTRIBUTION STATEMENT (of the abetract entered in Block 29, if ditterent from Report) 


18. SUPPLEMENTARY NOTES 


19. KEY WORDS (Continue on reverse eide i! Se and identity by block number) 
Assessment Selection 
Assessment center Simulation 
Development Situational tests 
Evaluation 





20. ABSTRACT (Continue on reveros sido tí necessary and identtfy by block number) 


The concept of the personnel assessment center is intro- 
duced, explained and evaluated by means of personal interviews 
and a literature review. Next, a proposed design for a proto- 
type assessment center for the Naval Postgraduate School (NPS) 
is offered along with the ancillary design of validation/eval- 
uation/feedback systems. Finally, a justification, listing of 
tasks to be performed, and anticipated results are provided. 


DD , AN 1473 EDITION OF t NOV 65 IS OBSOLETE 


(Page 1) SECURITY CLASSIFICATION OF THIS PAGE (When Date Entered) 


1 








es nn nn nn IM IMIM IMO 
SECURITY CLASSIFICATION OF THIS PAGE(When Data Enters: 


20. 


The reader is introduced to the concept through defini- 
tions, a brief history, and a comprehensive overview of the field 
Explanation and evaluation of assessment centers is accomplished 
by the reporting of interviews conducted by the writer, and the 
literature review. The assessment center design proposal for 
the NPS reflects the opinions of the writer, based on the researc 
conducted during the course of this project. 


Short and long term anticipated results are listed which 
cite potential benefits to individual participants, the NPS and 
the U. S. Navy. 





DD , Form, 1473 (BACK) 


Ian... 
S/N 0102-014-6601 SECURITY CLASSIFICATION OF THIS PAGE(When Data Entered) 


2 





The Personnel Assessment Center: 


A Review and Suggested Application 


by 


Galen Bruce Allen 
Commander, United’ States Navy 
B.S., University of Nebraska, 1955 


Submitted in partial fulfillment of the 
requirements for the degree of 


MASTER OF SCIENCE IN MANAGEMENT 


from the 


NAVAL POSTGRADUATE SCHOOL 
June 1974 





DUDLEY KNOX LIBRARY 
NAVAL OSTGRADUATE SCHOUI 
MONTEREY, CALIFO THA 273aí 


ABSTRACT 


The concept of the personnel assessment center is intro- 
duced, explained and evaluated by means of personal interviews 
and a literature review. Next, a proposed design for a proto- 
type assessment center for the Naval Postgraduate School (NPS) 
is offered along with the ancillary design of validation/evalu- 
ation/feedback systems. Finally, a justification, listing of 
tasks to be performed, and anticipated results are provided. 

The reader is introduced to the concept through defini- 
tions, a brief history, and a comprehensive overview of the 
field. Explanation and evaluation of assessment centers is 
accomplished by the reporting of interviews conducted by the 
writer, and the literature review. The assessment center 
design proposal for the NPS reflects the opinions of the 
writer, based on the research conducted during the course of 
this project. 

Short and long term anticipated results are listed which 
cite potential benefits to individual Ba ES, the NPS 


and the Navy. 
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I. STATEMENT OF THE PROBLEM 


During the past fifty years there have been dramatic 
advances in military "hardware" but relatively modest changes 
in personnel management techniques. Although the need to keep 
pace in weapons technology is not disputed, this writer believes 
that the potential return on investment available from improved 
personnel management has escalated sharply in the past few 
years--to the point where personnel research can now compete 
more ey with "hard" research for scarce defense 
resources. This belief is based on three factors: (1) The 
contention that people vice hardware are usually the decisive 
factor in combat; (2) The impact of the All Volunteer Force 
concept on the U. S. military; and (3) The implications of 
the Operational, Technical, Managerial System (OTMS) for the 
U. S. Navy. 

The first factor is mentioned (in a business vice military 
context) in Thompson's article [62] where he cites the view 
of D. C. Burnham, Chairman of Westinghouse Electric Corporation 
as follows: "Nearly everyone has the materials, the facilities, 
the techniques available to them, but how they use these things 
depends on their people." Admiral Thomas H. Moorer described 
the issue in a military context in an address to the graduating 
class at the Naval Postgraduate School on March 29, 1974. He 
reported that analyses of major naval battles of World War II 


disclosed that a majority of them were won by the side with 





technically, and/or numerically, inferior forces. His con- 
clusion was that the personnel involved had made the crucial 
difference. 

The second factor evolves from recent changes in the fabric 
of U. S. society which have brought increasing pressure to 
bear on the military services toward increased consideration 
of individuals. This is compounded by the reality of the All- 
Volunteer concept which has forced the military services into 
head-to-head competition with industry over the nation's human 
resources. The "sanctuaries" (draft, etc.), so long enjoyed 
by the military, have been abolished, and this has thrust the 
military manpower problem onto center stage. 

The third factor results, in part at least, from the other 
two which have begun to force changes in the traditional Naval 
Officer career patterns. The jack-of-all-trades, master-of- 
none Naval Officer generalist of yesteryear is finding it 
increasingly difficult to cope with the complex and varied 
problems of today's Navy. Bowling [6] expressed it thus: 

It has been a long-accepted fact that the Navy needs 
both good operators and good managers. What has not been 
accepted or fully recognized is that, in the current tech- 
nological, strategic, and tactical environment--given the 
complexities of nuclear ships and weapons systems, coupled 
with the relatively short span of an officer's career--it 
is no longer possible to develop both within the same 
individual. 

He further recommended that the CNO: 
establish the policy that all unrestricted line 
officers specialize in either Operations or Management, 
with the understanding that both specialties will provide 


equal but parallel paths to selection for flag rank in 
the unrestricted line. 
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Although Bowling's arguments have merit, they also have 
drawbacks, and his concept of sea-going operators and shore- 
based managers did not come to pass. Instead, the Operational, 
Technical, Managerial System (OTMS) was established in the 
summer of 1972. The concept, and the rationale behind its 
development were described, in part, as follows [107]: 

. . . Although there are several approaches which could be 
taken to provide specialization across the full spectrum 

of Navy endeavor, Admiral Zumwalt has firmly rejected the 
idea of going to a wet/dry Navy. (Writer's note: This 

was Bowling's proposal.) Navies of other nations have 
tried that approach and found it unsatisfactory. The 
Operational Technical Managerial System has as its goal 

the development of the broad sea and operationally-oriented 
unrestricted line officer blended with the technical exper- 
tise normally found in the restricted line and staff corps 
. . . Before a naval officer should consider becoming a 
career subspecialist (concentration within a subspecialty 
area), the officer should complete at least two operational 
tours . . . Through a system of evolution we are imple- 
menting . . . development and identification systems to 
meet the wide spectrum of Navy technical and managerial 
requirements. 

These factors illustrate the continuing and increasing 
need for selection (hiring, firing, promoting), development 
and placement within the U. S. Navy. In this macro context 
the need is for an integrated systems approach to personnel 
management involving many complementary subsystems. This 
study focused on one (proposed) subsystem: the personnel 
assessment center. The approach used was: First, to research 
the philosophies, structures and applications of the assess- 
ment center concept, and evaluate the concept in the ares of 
selection, development and placement; and, next, to put the 


assessment center in perspective within an "ideal" personnel 


management system, explaining the interfaces and relationships 
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between the assessment center and other subsystems; and, 
finally, to develop a research proposal for the design of a 
prototype personnel assessment center for the management cur- 
riculum of the Naval Postgraduate School. 
Hardesty and Jones [31] offer their opinion as to the 
potential value of the assessment center: 
Because the concern of the assessment center is with 


thewoxrpganizatisonis—lkeaders„of the future, the ultimate 
validity of this program . . . may well be measurable in 


the future success of the companies as contrasted to sim- 

ilar companies who do not make such an effort. The stakes 

are high, but on the other hand they always have been 

If the stakes are considered to be high in the business 

world, can they be otherwise in the military? The professional 
military man is acutely aware of the consequences of finishing 
second in combat. There will always be a need for good men 
and women--the problems are finding, developing and retaining 


them. This paper is devoted to the examination of one proposed 


solution to those problems: the personnel assessment center. 
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II. BACKGROUND 


A. DEFINITIONS 
A typical industrial assessment center is described by 
Byham [11] as follows: 


In these centers, specially trained managers (and occasion- 
ally psychologists) act as "assessors" who evaluate candi- 
dates for promotion--either into management or within 
management--on their potential and their areas of weakness. 
Groups of men pass through series of standardized exercises 
such as management games, in-basket tests, and leaderless 
discussion sessions, while assessors observe their behavior 
Closely. 


The assessors discuss each candidate's performance 
separately and then generate a comprehensive report on each 
Candidate which management can combine with current per- 
formance information as it sees fit. As well as identifying 
the men most likely to succeed, the assessment reports spell 
out the individual deficiencies of each candidate and suggest 
guidelines for management to use in developing him. 


The U. S. Army Infantry School (USAIS) Assessment Center 
[127] uses this definition: 


The term assessment center is used to describe a process by 
which candidates for a position are put through a series of 
standardized exercises designed to simulate the conditions 
of that position and to show if they have the skills and 
abilities necessary to perform it. 


The assessment process is conducted using a series of 
Situational exercises designed to measure leadership behav- 
iors and skills one level higher than the individual has 
performed. Officer candidates are cast in the role of 
2nd lieutenants, lieutenants as company commanders, and 
senior NCOs as platoon leaders. In addition, a battery of 
psychological tests are used to measure attitudes, mental 
ability, reading skills and interests. Peer ratings and 
self ratings combined with those of the trained assessment 
staff provide a trifocal view of each individual tested. 


As indicated by the USAIS definition, the term assessment 


center more properly refers to a process or program rather 
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‘than to a physical location. In fact, some "assessment centers" 
are taken to various corporation field sites for administration 


vice transporting candidates to an "assessment center" facility. 


bee HES TORY 

Various articles have traced the genesis of the assessment 
center to the Bible. Wilson and Tatge [70] reference the 
advice which Jethro gave to Moses on selecting his rulers: 

Moreover thou’ shalt provide out of all the people, able 
men, such as fear God, men of truth, hating covetousness; 
and place such over them to be ruler of thousands, and 
rulers of hundreds, rulers of fifties, and rulers of tens. 

Jaffee [35] cites Judges, Chapter 7 as the first use of 
Situational techniques: 

the Lord provided Gideon with a means of choosing the 
best among his men. "Bring them down unto the water, and 
I will try them for thee there . . . . Separate every one 
that lappeth of the water with his tongue, as a dog lappeth, 
him shalt thou set by himself; likewise every one that 
boweth down upon his knees to drink = 

From an initial force of 32,000 men, Gideon was able to 
select, by such techniques,an elite group of 300 and routed a 
numerically superior army. 

Prior to, and during, World War II the Germans used multiple 
assessment techniques in officer selection. The German efforts 
inspired the British War Officer Selection Boards (WOSB), which 
in turn influenced the program of the Office of Strategic 
Services (OSS) in this country. The other primary influence 
in the OSS program was the presence of Dr. Henry Murray of 
Harvard who brought with him concepts derived from his multiple 
assessments in personality research [97]. Assessment_of Men [99] 


by the OSS Assessment Staff is a detailed and very readable 


account.of this.effort. 
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All of the programs mentioned thus far were designed for 
the purpose of selection, however the wartime environment 
afforded neither time nor opportunity for proper validation. 

Following the war, only the British retained operational 
Assessment programs. Their Civil Service Selection Boards 
(CISSB) began validation efforts, while the WOSB evolved into 
the Royal Commissions Board (RCB) of today. 

As West Germany began rebuilding her armed forces, the 
Federal German Navy established an extremely sophisticated six- 
week assessment program for its officers [57]. This program 
has recently been merged into a tri-service staff officer 
course. 

Since the close of the OSS centers, the U. S. military 
has only dabbled sporadically in assessment center research. 
The latest effort was launched in 1973 when the U. S. Army 
Chartered a one-year evaluation of a pilot personnel assessment 
program. This pilot program was eStablished at the U. S. Army 
Infantry School (USAIS), Fort Benning, Georgia. | 

In addition to the British and Germans, the Australian 
Army and Israeli armed forces are known to have assessment 
programs. 

Although the U. S. military has lagged other nations in 
the use of assessment centers since WW II, U. S. industry has 
pioneered in the field. Dr. Dorie var Bray of AT&T set up an 
assessment program in the Management Progress Study in 
1956. Dr. Bray [7] describes this study as follows: 


Although the Management Progress Study was instituted as a 
long-term study without any expectation of immediate 
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mrertewieeresw@ltS =... . =, the mere conduct of the study 
and the reporting of gross observations on the total group 
of subjects have led to significant changes in certain 
Memseonnel practices ... Among them are... . the 
application of the assessment center method to the selection 
and development of managers. 

The Management Progress Study was designed as a ten-year 
longitudinal study--and indeed it was continued to fruition-- 
but the merits of the assessment center were quickly recognized 
and the first operational assessment center in the AT&T system 
opened its doors in 1958. Following AT&T's lead, other U. S. 
firms, such as : IBM, General Electric, J. C. Penney, Standard 
Oil (Ohio) and Sears, established assessment centers, and the 
program spread to other countries. 

The influence of the AT&T program has been considerable 


and most of the subsequent assessment programs bear at least 


partial resemblance to it. 


C. OVERVIEW 
Although all assessment programs emanate from the same 
basic concepts, they differ considerably along certain dimen- 
Sions, notably: 
1. Objectives (Selection, development, placement or research). 
Most of the applications are for selection, however a 
growing number have development as a secondary objective: 
2. Program Philosophies 
Assessment philosophies range along a continuum from 
psychometric to behavioral--or in another sense, from signs to 
samples. There are a few programs at either extreme, however 
most fall nearer the center and incorporate both signs and 


Samples. 
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3. Program Lengths 


These vary from one day to six weeks. The majority 
are five days; with two and one-half to three days of assess- 
ment followed by two to two and one-half days of evaluation 
and report writing by the assessors. 

4. Assessment Variables 

DingGemmest pmograms are selectiom-oriented, the big 
emphasis has been on stable variables. If development were 
a program objective, unstable variables would have to be included 
to identify those areas offering a viable training potential. 
Numbers of variables assessed vary from ten to fifty-two. 
Howard, in her survey of current assessment programs, [34] 
found the following dimensions to be the most popular: 

"(a) leadership, (b) organizing and planning, (c) decision 
making, (d) oral and written communications skills, (e) ini- 
tiative, (f) energy, (g) analytical ability, (h) resistance 
to stress, (i) use of delegation, (j) behavior flexibility, 
(k) human relations competence, (1) originality, (m) controlling, 
(n) self-direction, and (o) overall potential." 

5. Program Content (Instruments and techniques). 

The content of the program should be an extension of 
the program philosophy. The listing which follows will proceed 
from signs to samples and will be as inclusive as possible, 
vice representative of any single program: 

a Papew andapencile tests of attitudes, interests, 
mental ability, and achievement (math, reading grammar, etc.). 

b. Projective personality tests such as sentence 
completion, adjective check-lists, etc.. 
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c. Interviews by assessors. Some programs also have 
assessees interview other role-playing assessees to evaluate 
their interviewing ability. 

d. Written assignments such as essays. These may be 
solicited prior to arrival at the center. The topic may be 
of the candidate's choice, or assigned to him. 

e. Simulations may be either individual (the in-basket 
for instance) or “group “(such “as leadertess group discussions, 
management games, war games, etc.). 

The in-basket is a widely-used individual simulation 
with high face validity and a concomitant training effect. It 
consists of a series of notes, memos, and letters typical of 
the in-basket of the position for which the candidate is being 
evaluated. The scenario imposes a time constraint and forces 
the assessee to work alone without recourse to other individuals. 
He is provided ample information on the role he will play, (such 
as an Organizational chart, pertinent personal data on key 
associates, etc.) and is asked to put all of his responses in 
writing. He may delegate items to subordinates, take action 
himself, defer action until further information is obtained, 
schedule meetings, or write letters, as he sees fit. Most 
programs schedule an interview session following the in-basket 
where an assessor can question the candidate on the motives 
behind his actions. Evaluations of the in-basket range from 
informal subjective impressions tohighly-structured objective 
analyses. 

Leaderless group discussions typically assign the 


candidates differing points of view which they are to defend 
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individually. Following this discussion, they are asked to 
rank-order the positions. Since the points of view are designed 
to be equal in desirability, the candidate who is most per- 
suasive, influential and accepted by the group should emerge 
with the "winning" position. 
The various "games" used in assessment programs 
can provide evaluations of leadership ability, interpersonal 
skills, “decision «mekingsabidity, «organizational .ability and 
resistance to stress--depending on the particular objectives 
of the program. 
6. The Assessors 

several sources state that familiarity with the position 
sought by the candidates is the single most important attribute 
of an assessor. This probably explains why most programs uti- 
lize line managers two to three levels above the "assessment 
position" as assessors. A few programs employ clinical psy- 
chologists to interpret the projective tests, but very few have 
psychologists actively involved in the assessment and evaluation 
processes. Assessor/assessee ratios range from 1:1 to 1:4 and 
length of assignment as an assessor varies from one program 
(one to two weeks) to six to eight months or longer. Assessor 
training periods may be as short as a few days or as long as 
two to three weeks. The training may include lectures, films, 
etc., but the most effective and widely-used training technique 


is to involve the trainee in actual assessments on a "norm 


group." AT&T uses successful incumbents of the position being 
sought by candidates as the "norm group." This provides assessor 
trainees with two benefits: (1) a tangible example of performance 
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by organizationally defined "successes," and (2) an opportunity 
to err while learning without jeopardizing a candidate's career. 
A point upon which there is unanimous agreement is that well- 
trained assessors are an absolute prerequisite of an effective 
assessment program. 
7. The Evaluations 

After the assessments are completed, the assessors 
document their’observations. A»meetang -is»thensheld where can- 
didates' records are reviewed individually and ratings made 
by the assessors on the basis of all available information. 
Any disparities in the ratings are discussed further, however 
few organizations require consensus (e.g., that disparate 
ratings be changed). The final result may be "pass-fail" recom- 
mendations to line management, or merely reports of the assessors' 
evaluations. These reports are usually in narrative form (con- 
sidered best for use by laymen), however some firms prefer 
Meortless@onsisting of graphs or scales. 

8. Uses of the Data 

Most programs are selection oriented, however final 
selection decisions are not made by the assessors (with the 
possible exception of initial hiring situations where job 
performance data is not available, and the Federal German Navy 
program). In most programs, assessment evaluations are furn- 
ished to line management where it can be integrated with other 
information, such as performance appraisal data, as the managers 
see fit. In such cases (the majority) selection decisions lie 


solely with line management. 
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Development applications are few, but increasing in 
numbers. They generally provide the candidate with knowledge 
of his strengths and weaknesses. The "ultimate" outcome would 
be a comprehensive development plan specifically tailored to 
the individual. 

Programs ‘with placement as an objective are rare. The 
Sohio program includes placement as an objective and is described 
‘by sFinklevand.Jones [85]. 

9. Feedback to Candidates 

Feedback is a part of most programs--including selection 
oriented ones--even though it represents a developmental function. 
It may be provided to all candidates, or only to those who 
request it. Feedback may be given by lay assessors, psychologists 
or line managers--as the organization prefers. It is usually 
oral, confined to observed behaviors and test performance, and 
provides the candidate with recommendations for a personal 
development plan. 

10. Summary 

Bender [3] provides an excellent overview summary. He 
conducted a survey of organizations operating assessment centers 
and obtained data from thirty-four operational programs. The 
results of his survey are presented in the following figures 
and provide a good picture of the current character of assess- 


ment in the United States and Canada. 
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Figure 1. 


Primary Evaluation Devices Used by Companies 


Operating Assessment Centers 





In-basket exercises used. 
Business..games exercises used. 
Assigned roles 
Nonassigned roles 
Leaderless group discussion exercises 
used. 
Assigned roles 
Nonassigned roles 
Films used. 
Video tapes used. 
Psychological tests used. 
For the most part, assessment devices 
are locally produced. 
For the most part, assessment devices 
ro ap Eras cate tornally. 
An in-depth background interview is 
given to assessees. 


Source: Bender [3] 
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Figure= 2. 


Operating Procedures Followed in Administration 
of Assessment Centers 





Assessees are nominated by supervision. 

Assessees nominate themselves for 
-assessment.” 

A full-time director administers the 
assessment center. 

A part-time director administers the 
assessment center. 

The assessment center director is an 
industrial psychologist. 

Females are assessed., *¥* 

Minorities are assessed, ** 

Homework is assigned. 

Assessee's performance is considered 
as pass or fail. 

Female assessors are used. 

Assessment is conducted off-site and 
away from the work environment. 

Assessors rotate in viewing assessees 
during evaluation. 

Assessee performance is video-taped for 
mse in evaluation. 


Re-assessment of assessees is permitted. 


A checklist for evaluation is used by 
assessors during evaluation. 

Peer evaluations are made by assessees. 

Assessment center has been validated 
against performance criteria. 


*Two companies considering for future evaluation. 
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**Only one organization indicated that it had not yet assessed 
either females or minorities, but it added that no restrictions 


existed for not doing so. 


Source: Bender [3] 





Figure 3. 


Operating Characteristics of Assessment Centers 


Number of assessees evaluated when your assessment center operates: 


ee 
Response frequency 1 2 Ze il 2 3 


Levels‘assessed‘by”your vassessnent-center: * 
mst Line Middle management Upper management Newly hired 
24 17 - 3 7 


Number of days your assessment center operation takes: 
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Total number of evaluation exercises used during the operation 
of your assessment center: 


10 11 
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15 40 
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2.6 7 
Response frequency 6 3 4 


Number of years your assessment center has been in operation: 


0-1 1 2 4 5 6 8 10 15 
2 1 


5 
Response frequency: A A | I g 


Number of employees you have assessed to date:+ 


0-500 500-1000 1000-3000 75000 
25 5 Z 1 
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Response frequency: 


*Several companies indicated that their assessment center was 
ard for evaluation of more than one level of supervision. 
+0ne center was not yet operational. 


Source: Bender [3] 
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Figure 4. 


Evaluation Parameters Most Frequently Observed in 


Assessment Centers 


Parameter 


Impact 

mnergy 

korsefulness 

BeTteptıon 

Creativity 

Range of interests 
Salesmanship 

Listening skill 
Risk-taking 

Independence 

Management control 

Use of delegation 
Problem analysis 
Leadership 

Oral communication skills 
Oral presentation skills 
Written communication skills 
Organizing and planning 
Decision-making skills 
Inner work standards 
Resistance to stress 
Attitude toward peers 
Attitude toward subordinates 
Attitude toward superiors 
Self-evaluation 
Behavioral flexibility 


Source: Bender [3] 
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Figure 5. 


Uses Made of Performance Data 
Generated by Assessment Centers 


Assessees are provided immediate 
feedback on their performance. 

Feedback is given orally to the 
assessee. 

Feedback is given in writing to 
the assessee. 

Feedback is given through the line 
organization. 

Feedback is given assessee only by 
assessment center personnel. 

Assessee's evaluation is made 
available to top management. 

Evaluation results are used to 
prepare a formal plan of develop- 
ment for the assessee. 

A follow-up of an assesse's continued 
development after assessment results 
are reported is a routine function of 
assessment center activities. 


Source: Bender [3] 
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Figure 6. 


A Summary of Key Characteristics of Assessment Centers 


In-basket exercises are used. 

Games and leaderless group discussions (with both assigned and 
nonassigned roles) are used. 

Psychological tests are used in conjunction with simulated 
exercises. 

Assessment instruments are locally produced. 

In-depth interviews are conducted. 

Assessees are nominated by supervision. 

Four simulation exercises are used during evaluation. 

Assessment centers are operated by part-time directors. 

Females are assessed. 

Minorities are assessed. 

Homework is assigned. 

Assessment is conducted away from the work environment. 

Assessors rotate in viewing assessees during evaluation. 

Re-assessment is permitted. 

Peer evaluations are made by assessees. 

One assessor for one assessee is used. 

A maximum of six people are assessed at one time. 

Assessors are two organizational levels above assessees. 

Assessment centers operate for three days. 

Assessee's performance is rated, not ranked. 

Rating systems are based on research. 

Parameters assessed depend on the purpose of the assessment 
center. 

Assessees receive immediate oral feedback on their performance. 

Assessees receive written feedback on their performance. 

Feedback is disseminated by assessment center personnel. 

Assessment center results are used for formal development 
plans. 


Source: Bender [3] 
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IIT. APPROACH 


A. INTERVIEWS 

Working within time and fund constraints, four POE 
visited. These included two military (U. S. Army) applications, 
an AT&T (Pacific Bell Telephone) assessment center and an 
assessor training session for a one-day behaviorally-oriented 
program. A brief summary of the visits with lists of signifi- 


cant aspects of each site and application follows. 


1. United States Army War College (USAWC), Carlisle Barracks, 
Pennsylvania 


This visit afforded the opportunity to interview indi- 
Viduals who were involved in the USAWC "Study of Leadership for 
the Professional Soldier" [128], [129], [130], and [131]; namely, 
Dr. Donald D. Penner and LTCOL Dandridge M. Malone. In addition 
to providing copies of the references cited above, and amplifying 
information on the study, they outlined some of the Army's other 
activities which preceded the USAIS assessment program. 

The influence of the Center for Creative Leadership in 
Greensboro, North Carolina (Olmstead, et.al.[122] refers) was 
explained. When the Army became seriously interested in assess- 
ment centers, a few years ago, twelve Brigadier Generals were 
tionsh the Center for Creative era to determine 
their reaction to an assessment program. This program was 
purported to contain several paper and pencil tests and one 
complex simulation. The simulation is of particular interest. 


It featured a scenario set in the 31st century, and required 
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candidates to cycle through six defined leadership roles. The 
futuristic setting was chosen to eliminate any possibility of 
bias from previous candidate experience. The entire simulation 
was observed by assessors through "one-way" mirrors, and also 
video-taped. At intervals, action was halted and candidates 
were asked to record their perceptions of the exercise at that 
point. This simulation lasted an entire day, and was followed 
by a’detafled"fecdback which “stressed: the~companison between 
the individual's self-perception and the perception of him by 
others. 

Because this program is oriented toward research and 
self-development, the Center does not provide any assessment 
information to a candidate's parent organization. It does, 
however, provide transcripts, video tapes and other data 
obtained, directly to the candidate. The twelve Brigadiers 
who underwent assessment were reported as enthusiastic in their 
endorsement of the program as a vehicle of personal inventory 
and development. 

LTCOL Quay Snyder [124] offered much detail on the 
AWC Inventory and Assessment. His review of relevant instru- 
ments and information on source material was especially valuabie. 

Probably the greatest benefit from the AWC visit derived 
from the constructive criticisms of this writer's concepts of 
an assessment program, and the alternative suggestions offered. 
Though too numerous to acknowledge individually, there were 
many seeds which took root and flourished during the ensuing 
research period. The results provide much of the substance 
of this thesis. 
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some OL the more significant factors were: 

a. The concept of more than one assessment during the 
course of a career. 

b. The "systems approach" to personnel management 


which views assessment centers as a subsystem of the larger 


System. 

c. The USAWC Leadership Study. 

.d. ..An.assesement. program. applied.in.a.military environ- 
ment. 

e. An assessment program devoted exclusively to develop- 
Ment., 


2. United States Army Infantry School (USAIS) Assessment 
Center Pilot Peano yc cer 


This operation represents a current military application 
or the "ATST model," e.g., a program which utilizes a MIx Of 
paper and pencil tests and experiential instruments. The visit 
afforded interviews with the director, COL Wallace Veaudry; 
assistant director LTCOL Kenneth Smith; staff members, MAJ John 
Campbell and MAJ Richard Davis; and Dr. Kay Smith of the Army 
Research Institute (ARI) Field Unit which is co-located with 
the assessment center. 

The USAIS assessment program entails forty hours of 
assessment in a three-day period. This heavy schedule is a 
result of the inclusion of "competing" instruments for evalu- 
ation. At the completion of the evaluation, the program will 
be factor analyzed and thoseinstruments with little or no 
unique contribution will be eliminated. This should stream- 


line the program to a more realistic schedule. 
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A unique feature of this visit was the fact that 
current staff members had been involved in the design and 
implementation as well as the operation of the assessment 
program. Because of this, staff members were exceptionally 
understanding and knowledgeable concerning the focus and 
requirements of this study. Additional background on U. S. 
Army assessment center research was provided, and an assess- 
ment center research program conducted’ at Port McClellan, 
Alabama circa 1965 was reported. It was described as a very 
expensive operation where candidates were assessed one at a 
time, thus the assessor/candidate ratio was something like 
meee In contrast, the present USAIS ratio is about,11:18, 
which is more representative of ratios in current industrial 
essessment programs. The use of video tape, and the structure 
and content of the counseling feedback were other facets of 
the program worthy of mention. 

In A the following items were of particular 
interest: 

a. Information concerning assessment center design, 
and implementation--as well as operation. (Olmstead et.al. 
[122] pertains to design of this program). 

b. Simulations tailored to military sıtuations. 

c. Use of video-tape in analysis and counseling. 

d. Feedback techniques and counseling training. 

e. Differences between programs, reflecting different 
levels of candidates and positions being assessed for, e.g., 


differences in course content between the following: 
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Course Candidate Level 


Officer candidate course ES 
Advanced NCO course E6-E7 

Basic Officer course ET 
Advanced officer course CPT 

f. Assessor training and employment. (Assessors are 


assigned for a normal tour of duty--about two years--therefore 
are long-term in contrast with short-term assessors prevalent 


in industry programs). 


3. Dr. Cabot L. Jaffee, Florida Technological University, 
optando. Florida 


The visit coincided with an assessor training session 
which Dr. Jaffee was conducting for selected civil service 
personnel of the Naval Training Center, Orlando. The training 
was in preparation for a one-day assessment program composed 
of behavioral instruments (simulations, etc.). This provided 
an opportunity to observe assessor training which included the 
administration and scoring of the in-basket exercise included 
in the assessment program. 

Dr. Jaffee has extensive experience in the field of 
assessment centers and has contributed much to the literature 
[16], [35], [36], [89] and [90]. Because of this, his comments, 
criticisms and suggestions were particularly valued. 

Highlights of the visit were: 

a. Observation of assessor training. 

b. Views of a prominent consultant in the field of 


assessment centers. 
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4. Personnel Assessment Review (PAR) Program, Pacific 
| Telephone Company, Oakland, California 
Because of the great influence of the "AT&T model" on 
Subsequent assessment programs, a visit to a Bell system program 
was given high priority. During the interview attention was 
focused on operational aspects rather than design or theory. 
Salient points of the program included: 
a. Clinical combination of variables in the evaluation 
process. 
b. Practice of having candidates' supervisors observe 
the evaluation process. 
c. Pre-assessment interviews done by assessors at 
candidate's organization. 
d. Feedback to candidates done by assessors at candi- 
date's organization. 
| e. Practice of mixing men and women in groups vice 


assessing them separately. 


B. LITERATURE REVIEW 
1. Overviews 

This section is provided as a guide for the reader who 
seeks additional overview information. It does not contain 
fesereptive data, per se, but rather directs the reader to 
selected articles in the literature. Those readers not 
desiring such referral advice may proceed directly to section 
ITI-B-2, Current Applications. 

Several articles in the literature provide excellent 
overviews. These will be cited in the following order: (1) 
General articles covering a wide range of programs, (2) Articles 
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dealing with a particular type of program (e.g., the five-day 
program patterned after the AT&T model, etc.), (3) Articles 
dealing with specific organizational programs. 

An article in Business Week [68] provides a broad 
survey of the field from one-day assessment programs, to the 
"five-day" programs employed by larger firms such as AT&T, 

J. C. Penney, Standard Oil (Ohio), etc. and gives some rough 
Cost figures. Howard -[:34] -presents -an-excellent aggregation 
of information from the entire field. Cohen and Jaffee [16], 
although primarily concerned with government applications, 
cover most of the significant developments in the field. They 
also provide some predictions on future growth and changes in 
government assessment center applications. 

Slevin [59] describes a program typical of those found 
in the major firms employing assessment centers (e.g., AT&T, 
Sears, IBM, General Electric, J. C. Penney, Standard Oil of 
Ohio, etc.). This article has a particularly comprehensive 
coverage of the subject, touching briefly on most of the 
Coone opic al areas. This writer considers Slevin's predic- 
tions for the future of assessment centers to be particulariy 
perceptive and well-founded. Other authors writing on assess- 
ment programs of this type include: Eok [1], Byham [il], 
and Byham and Pentecost [12]. 

McConnell [43] describes a one day assessment program 
in a small company. McConnell [44], [45] and McConnell and 
Parker [46] discuss the one day "package" assessment program 


available from the American Management Association (AMA). 
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Information on the AT&T (Bell System) programs is 
abundant in the literature. Articles providing general data 
include Jaffee [35], Jelks [37], and Wikstrom [69]. 

The Standard 0il of Ohio (Sohio) program is treated 
in detail in the book by Finkle and Jones [85]. Chapter four 
of the book contains the overview. This program is also 
described by Hardesty and Jones [31]. 

Veaudry and Campbell [65] describe the USAIS assess- 
ment center pilot project, while Snyder [124] covers the USAWC 
developmental assessment program. Information on foreign 
military programs are found in the USAIS unpublished report 
of a trip to the RCB [126] and Schmahling's [57] article on 
the Federal German Navy's program. 

2. Current Applications 

a. Industry 

It is difficult to obtain an accurate estimate of 
the number of assessment centers in existence, however the 
literature provides a picture of increasing growth in this 
area. Pomerleau [53] notes: 

+ . AT&T affiliates . . . currently operate over 50 
centers throughout the country processing upwards of 
10,006 candidates a year at an annual cost of $5 million 

contributors to professional journals . . . in the 

last five years, have written some fifty-four articles 
and reports describing and evaluating the assessment 
center method. 

McConnell [45] describes the record of a one-day 
package assessment program: 

In 1970, at the end of a five year research and develop- 
ment project, the American Management Association (AMA) 

introduced a multimedia assessment centre programme 

in the three years since its original publication, over 


150 organizations have used it for assessment of their 


management development needs. 
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Byham [11] offered this estimate of assessment 
center growth, circa 1970: 


To my knowledge, twenty companies have been responsible 
for assessing more than 70,000 candidates in the last 

ten years, but at least 100 more companies are developing 
centers or are in an advanced stage of center planning. 
Many others are "looking into the idea." As an indication 
of this interest, I might cite the fact that more than 

200 company representatives attended conferences on the 
assessment center method during 1969. 


Two articles have been selected to provide detail 
on current industrial assessment programs. The first concerns 
a one-day "package age eT offered by the American Management 
Association (AMA). Such a program puts assessment techniques 
Within reach of smaller companies who cannot afford to develop 
and maintain programs of their own. The article (by McConnell 
[45]) describes exportation of the AMA program to Europe and 
is written with a British flavor: 

The assessment centre programme consists of 5 basic parts: 


Chairman's training--A representative of the organization 
spends one week at an intensive training programme conduc- 
ted by Management Centre Europe to obtain all of the know- 
ledge and skills necessary to conduct all phases of the 
programme within his own organization. 


Assessor training--Upon returning to his organization, the 
trained chairman selects five existing managers to be 
trained as evaluators in the programme. These managers, 
once trained, are referred to as assessors. Their training 
consists of a 28 hour multimedia. training course in which 
they are taught the.skills and techniques necessary to 
serve successfully as assessors. Included in this training 
is a measuring device to insure their skills are at a level 
aumwhichethey can successfully serve as assessors. 


Management simulation workshop--Once the assessors are 
trained, a management Simulation workshop is conducted. 
iis as) One day in length and consists of eight exercises. 
Twelve participants are selected to go through the eight 
exercises and are observed in these exercises by the 
assessors. The assessors' role in the workshop ıs tc 
observe and record the behavior of the participants. Fol- 
lowing the one day workshop, the participänts are allowed 
return to their jobs. 
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. Evaluation meeting--The assessors and the chairman meet 
following the management simulation workshop to report 
and evaluate the behavior of each individual observed. 
This final evaluation yields the report on each individual 
participant in 12 specific management abilities, overall 
management ability, and the individual's organizational 
potential. This evaluation meeting requires approximately 
Ze OUrS . 


Feed-back--At a feed-back session between the chairman 
and “each participant, the results of the centre are com- 
municated to the participant, compared with his own self 
evaluation, and specific development plans for the par- 
ticipant are established. 


The second article (Byham [11]) describes the most 
Widely used format: the five-day program patterned after the 
AT&T model. Although the following material was based on the 
J. C. Penney program, it is generalized enough to typify 
practices of many of the major firms. 
Sunday 


Six management assessors meet at a conveniently located 
motel and organize materials for the week's activities. 
Late in the day, twelve candidates, all of them of com- 
parable rank in the company, arrive and settle in. 


Monday Morning 


Period 1: After orientation announcements the candidates 
are divided into teams of four, for participation in a 
management game. Each team is given a limited amount of 
Capital to purchase raw materials, make a product, and 
sell it. The raw materials are usually tinker-toy parts 
which can be assembled into a variety of products of dif- 
ferent complexity, each of which has a different, pre- 
Specified market value. 


The players must first decide how to invest their 
Sete to Maximize profits and then organize the pur- 
chasing, manufacturing, and selling operations. Assessors 
observe the players for signs of leadership, organizational 
ability, financial acumen, quickness of thinking, and 
efficiency under stress. 


Suddenly the players are notified that the prices of 
the raw materials and the products have been radically 
Changed, requiring drastic redeployment of capital and 
extensive operational reorganization. As soon as they 
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have regrouped, these prices are abruptly changed again. 
The actions the players take allow the assessors to esti- 
mate their adaptability. 


The game is then halted, and each candidate is asked 
to write a report evaluating his own performance and that 
of his fellow players. 


Period 2: The candidates are divided into groups of six. 
While one group takes written psychological tests, the 
members of the other group are interviewed individually 

by the assessors. The assessors have been provided with 
detailed background information on each man, and they use 
this to probe for evidence of drive, motivation, and sense 
of.self-development. This Assessment Interview, so called, 
is ordinarily the only exercise in the assessment process 
that focuses on the candidate's past behavior. 


Monday Afternoon 
Period 1: The testing and interviewing groups are reversed. 


I oa: in two leaderless groups of six, the candidates 
join in discussion of a promotion decision. Here the can- 
didates play the role of supervisors brought together on 
short notice by their boss to pick one man from a pool of 
six for advancement. Each candidate receives the file of. 
one of the men in the pool, whom he is then to "champion" 
for the promotion. After each candidate has studied his 
protege's folder, the group meets for an hour's discussion 
to choose the man it will recommend. Assessors observe 
the candidates' exchanges in the meeting for signs of 
aggressiveness, persuasiveness, expository skill, energy, 
Me ty, self confidence, and the Tike. 


Alternative exercise: In leaderless groups of six, candi- 
dates discuss the 20 most critical functions of a inanager 
and list them in order of importance. (This forces them 
to think about the qualities on which they are being 
assessed.) Each group then chooses a spokesman who pre- 
sents the list and the rationale behind it to the whole 
group of assessors and candidates. 


Monday Evening 


Each candidate receives material on how best to conduct 
employment interviews and also the resumé of a job appli- 
Bent. He studies these for use in one of the exercises 
on the following day. He may also receive special phone 
calls--for example, the Irate Customer Phone Call. 
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Tuesday Morning 


esday morning is devoted to the In-Basket Exercise. This 
Simulates the experience a candidate would have if he were 
suddenly and unexpectedly promoted a grade or two and 
arrived at work one morning to find his in-basket full of 
unfamiliar material typical of the sort he would then have 
to handle. He is instructed to go through this material 
and deal with the problems, answer the inquiries, request 
additional information where he needs it, delegate tasks 

to proper subordinates, and generally organize and plan 
just as he would if he had actually been promoted. 


Tuesday Afternoon 


Period 1: The candidates conduct the employment interviews 
for which they prepared the night before, each interview 
taking place in the presence of an assessor. The appli- 
Cants are college students who have been especially trained 
in the applicant's role. The interview itself lasts 
roughly halfan hour, after which the applicant leaves and 
the assessor quizzes the candidate to determine what in- 
Sights he has obtained about the applicant. 


Peapod 2: —thesmext exercise is the resolution of disci- 
plinary cases. In groups of four, candidates decide how 
to allocate their time between three such cases and then 
decide the cases themselves, within one hour. This exer- 
cise provides the assessors with information on a candi- 
date's appreciation of personnel problems and his 
sensitivity to subordinates' views of events and actions, 
as well as insight into his behavior within a group. 


Alternative exercise: The candidates are assigned roles 

as city councilmen who meet to allocate a $1 million 
federal grant to the city departments. Each "councilman" 
interprets a briefing document provided by a city agency-- 
the police department, sanitation department, water depart- 
Ment, and so Oon--and tries to get aSamuch ofthe grant 
allocated to this agency as possible. Again, effective 
discussion is limited to one hour. 


Tuesday Evening 


Detailed data on a company was provided to all the candi- 
dates. Each is asked to examine its financial and 
marketing situation from the viewpoint of a consultant 
and to prepare a written recommendation for its board of 
directors on the future expansion of a particular part 
Samet. product line. 


At the same time, also in preparation for the next 


day's activities, the assessors study the results of the 
eamaagaates' In-Basket tests in detail. 
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Wednesday 


Period 1: Four groups are formed, each consisting of three 
candidates and an assessor. Each candidate takes his turn 
presenting his oral analysis of the company data studied 
the night before and submitting written recommendations. 


Period 2: These three candidates work together for an 
hour to reconcile and consolidate the recommendations. 


Period 3: The In-Basket Interview follows, in which an 
assessor discusses with a candidate the various actions 

he took. This further defines each man's grasp of typical 
problems and opportunities. 


Wednesday Afternoon 


In a final group session, the candidates rate each other 
and ask any.questions they may have. They then leave for 
home. 


Wednesday Afternoon to Friday 


The assessors discuss the candidates and prepare their 
ratings and reports. | 


Throughout all the exercises, the assessors have been 
rotated so that as many as possible have had a chance tc 
observe each candidate closely. Thus, in these discussions, 
the assessor who conducted Jones's personal interview 
summarizes his background and his own impressions of his 
behavior in the interview; next the assessor who checked 
what Jones did in his In-Basket Exercise and interviewed 
him on it presents his impressions; and so on. Each 
assessor attempts to keep these descriptions nonevaiuative 
and objective. 


Only when all the assessors who have observed Jones 
have spoken does the group begin to judge his behavior 
from the viewpoint of his management potential and the 
directions in which he needs to develop. After they have 
reached a consensus, they prepare a final report. 


Within two weeks a manager who has had experience as 
an assessor meets with Jones to communicate the results. 
In this meeting he lays stress on the areas in which Jones 
needs to develop himself and encourages him to set appro- 
priate goals. 
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b. Military 

Four sources have been selected to provide samples 
of widely divergent military programs. The first, Snyder [124] 
describes Inventory and Assessment at the United States Army 
War College (AWC). Although this program is atypical of the 
assessment programs described thus far, it is of interest 
Peemsc it 15 purely development-oOriented. Instead of a tightly 
‘scheduled«program of a few days duration, the assessment is 
interspersed throughout the academic year (September - June). 
Another difference is that faculty coaches or counselors replace 
the assessors previously described. Each coach is assigned 15 
Students to counsel throughout the school year. The program 
employs five assessment techniques, e.g., Baseline, Inventory, 
Optional Personal Inventory, Professional Self-Assessment, 
Experiential Assessment, and Value-Gained Inventory. They are 
described as follows: 

OE Baseline Inventory. A questionnaire is mailed 
to the student prior to his arrival at the AWC. The student 
indicates his competence in a variety of areas related to the 
Mnn culum and returns the questionnaire, whereupon the results 
are compiled. Feedback is provided which shows the class range 
and mean on each item. In this way, the student and the school 
Pin calor individui eurricula to best advantage. Future 
plans “are to include additional instruments in this phase, such 
tn Graduate Record Exam, to assess academic potential, 
wa er tests for reading comprehension, listening compre- 


Mencion, writing skills, etc. 
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(yeep tron bersona] Inventory (OPI). Research 
on executive development is said to identify a need for 
reassessment of personal and professional goals which occurs 
when an individual iS about age forty and two-thirds of the 
way through his career. The OPI was designed to meet this 
need. <A battery of tests is available to the student which 
can be administered and analyzed by his faculty coach. The 
battery includes»measures of ES, reading and writing 
abilities, attitudes, personality traits, needs, vocational 
interests, etc.. 

(3) Professional Self-Assessment. This item 
provides feedback in three ways: first, to the student, second 
to the instructor, and third to the institution. The need for 
such a feedback system is underscored by the fact that the 
AWC does not utilize an academic grading system. The self- 
assessment consists of a final examination administered to the 
students after which they are provided with solutions to enable 
self-grading. The unsigned exam papers are then given to the 
instructor for analysis by him and the institution. 

(4) “Experiential Assessment. A group ™probren— 
solving exercise is conducted by a team of students lead by 
wa uy coach. “Upon completion, exercise performance is 
critiqued by a panel of experts. Each student is provided 
feedback on his performance by the faculty team-leader, his 


peers and the panel of experts. 
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(5) Value-Gained Inventory. Essentially, admin- 


istration of alternate, but equivalent, forms of the Baseline 
Inventory at the end of the academic year. Statistical analysis 
would be utilized to determine statistically significant changes. 
Peeults could then be provided to students and faculty, and 
also used in curriculum development. 
The AWC program was initiated in academic year 
1973 - 1974,-therefore it ís too early.for an evaluation, how- 
ever,. the implications for the Naval Postgraduate School (NPS) 
are obvious and would suggest a close and continuing follow-up. 
The second military application to be considered 
is the pilot assessment center project at the United States 
Army Infantry School (USAIS), Fort Benning, as reported by 
Veaudry and Campbell [65]. This program is patterned after 
the "AT&T model" and entails three days of intensive assess- 
ment. There are actually three separate programs: one for 
senior noncommissioned officers (NCOs), a second for lieutenants, 
amd asthimd for captains, however, they follow the same basic 
philosophy and share many of the commercial instruments, e.g., 
paper and pencil tests, inventories, etc.. The programs are 
based on the "Leadership for Professionals" study conducted 
by the Army War College. Ten exercises were designed for 
each program, and combined with batteries of paper and pencil 
tests. The resulting assessments include writing exercises, 
in-baskets, leaderless group discussions, war games, field 
exercises, simulations of organizational problems in the 


field and three types of interviews. 
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Evaluation reports consist of numerical ratings 
and narrative profiles describing candidates' strengths and 
weaknesses. The narrative profiles include a one page summary 
Birabılıties and an eight to twelve page detailed description 
based on the observations of the assessors. 

The Be assessment center project was chartered 
for one year and given the following missions: 


Pest t herassesenen tweoneep tras sa™readersisip—and career 
development technique. 


Test the assessment concept as a precommission screening 
device. 


Determine the feasibility and applicability for Armywide 
use of the concept and techniques developed under the first 
Ewo Steps. 


As with the AWC program, the assessment center project has 
Significance for the Navy and warrants close scrutiny. 

The British Royal Commissions Board (RCB) is one 
of the oldest military assessment programs in continuous oper- 
ation. The USAIS Assessment Center Project trip report [126] 
provides the following overview of the RCB: 


The RCB . . . has the mission of selecting candidates 
with the potential qualities of character, ability and leader- 
Ship necessary for a career as an officer. All candidates 
for either Short Service, Limited Service or Regular Com- 
missions must be evaluated by the RCB prior to any pre- 
commissioning training (Sandhurst or University course). 


The RCB has a President (Major General), three Vice 
Presidents (Brigadiers), a Training Officer (LTC), (and 
twelve assessors) (six Lieutenant Colonels and six Majors), 
all assigned ontwo year tours. The Board has the cgpa- 
bility of assessing 48 candidates each week, organized 
mmo groups of 8 each. Each group, therefore, is directly 
Supervised by a Major, whose principal function is detailed 
observing, and a LTC, whose functions include observing and 
conducting in-depth interviews of each of the 8 candidates. 


Although there is no permanent psychologist assigned 
to the Board, the Board draws on the professional advice 
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of psychologists assigned to the Army Personnel Service, 
and a civilian firm of consultants. In addition, each 
group of candidates receives an educational/intelligence 
rating from an Education Advisor attached to the RCB from 
the Army Education Board. 


The criteria used for evaluating each candidate are 
(1) Training Potential, (2) Applied Ability, and (3) 
Character. Within each major criterion, the following 
facets are considered: 


on Training Potential 


(a)-Officer intelligence rating (IQ) 
(b) Educational standard 
(c) Self-expression (written and oral). 


Ihe Training Potential is assessed by the Education 
Advisor, based on records (achievement vs. opportunities) 
and tests conducted at the RCB. The candidate's overall 
rating 1S given a numerical value as follows: 


Strong Good Adequate Limited Weak 
10,9 8 7 6 Coes rel 
(PARADA ed Ability 


(a) Planning ability 
KeiepPractical ability 
(c) Physical ability. 


(3) Character 


(a) Coolness (ability to relax) 

(b) Initiative 

(c) Sense of urgency 

(d) Determination 

(e) Military compatibility 

(f) Sense of responsibility 

(g) Sense of awareness (breadth of knowledge) 
(h) Liveliness 

(i) Dominance 

D Ouality of character 

(k) Range of personal relations (ability to mix). 


The schedule for each session is divided into Parts I 
and II. Due to the large number of candidates, it was felt 
necessary to establish an initial screening process to 
reduce the number of candidates down to 48 for the actual 
assessment. The schedules are as follows: 
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PART I 


Monday Tuesday 
Intelligence test Part I Board sits 
Current affairs Eliminated depart 
Essay 

Interviews 


Evaluation of reports 


PART II 
Tuesday Wednesday Thursday 
Group “discussion Wreetren=pnegect s Group race 
Group task Individual tasks Final Boards 
Group race Obstacle course Results posted 
Interviews Lecturettes 


The Majors and LTC's are rotated each session to prevent 
bias, and their recording of information is done independently. 
The Major simply observes; he does no interviewing, neither 
does he see any written reports or references on the candi- 
dates. In addition to observing, the LTC conducts interviews 
in depth, but also does not see any reports or references. 

A preliminary finding is determined after the first day with 
the Brigadier reconciling the differences between the Major 
and the LTC. The reports and references have been previously 
studied and graded by the Brigadier. 


Based on the preliminary findings (Straight up, Doubtful 
Moe Doubtiul, Doubtful down, Straight down), the President 
of the Board will then conduct his own interviews, observe 
candidates as they (perform) executive leadership "tasks," 
and attend those final boards he feels are critical. The 
Brigadier makes the final, overall assessment rating, 
wp ect to the approval- -of the President. The rating is 
either Pass, Fail or Return; with copies forwarded to 
Sandhurst anid: the Ministry of Defense. Historically, the 
Board's pass rate has been some 19 percent for Sandhurst 
(Regular Commission) candidates, 11 percent for Limited 
Service candidates and 15 percent overall. 


The final military assessment program to be des- 


cribed is the recently discontinued Staff Officer and Selection 


Course of the Federal German Navy described by Schmahling [57]. 


This six-week assessment program was in existence from February, 


1959 to December, 1973. During its fifteen years of operation, 


2287 officers underwent assessment. 
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The basic objective of the program was selection 
Te Officers. In 1966, the selection category of Admiral 
Staff Officer was added. The criteria used consisted of three 
Dot variables pertaining to personality, mental ability and 
problem-solving capability. It became evident that personality 
traits were poor predictors and the program was finally based 
entirely on assessment of mental abilities. 

As of 1968, -an overall -score has .been obtained by 
combining scores from eight items, each with equal weight. 
These eight items include: four written tests, three essay- 
emcees, a term paper, four oral tests, two in-basket simu- 
lations, a war game and an oral presentation. During the years 
the program underwent constant refinement, and efforts were 
made to control for bias due to the differing backgrounds 
of the candidates. Tests were selected and designed to afford 
m sponses from differentəlevels»of candidate experience and 
sophistication. Emphasis was on methodology vice recall of 
previously learned knowledge. 

Despite the fact that the program was selection- 
Oriented, a number of developmental effects were noted. For 
most officers, the return to academic pursuits after years of 
operational service was a challenge. Another stimulus was 
provided by the high standard of performance demanded and 
obtained which produced a wholesome competition between candi- 
Piece hard factor was the knowledge that assessors were 
ever-present and observing every nuance of the candidates’ 


performance. 
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The only requirement Cited.for assessors was that 
one assessor for each group of ten candidates had to have com- 
pleted Admiral's Staff training. Thus, assessors typically 
possessed broad operational experience and were expected to 
have a sense of responsibility for their fellow officers and 
the capability of making pragmatic decisions. The practice of 
multiple assessment was seen as easing the burden of responsi- 
bility for the assessor and increasing the candidate's confi- 
dence in the final rating. 

With few exceptions, the candidate received only 
the final result upon completionof the course. Two reasons 
given for this practice were to save time and to prevent 
weak candidates from giving up midway through the program. 
(This writer does not consider either of these very compelling 
and speculates that the organization had other reasons for 
withholding feedback). Schmahling asserts that the lack of 
feedback was rarely criticized but that many candidates felt 
that past performance evaluations should have been considered 
in the final rating. 

The Federal German Navy program has been replaced 
by a tri-service assessment center, however, as the author 
points out, the fifteen years of assessment experience is not 
lost, but can be profitably applied to the new assessment 
program. 

3.. Advantages (Support for the concept) 
Three articles have been selected which state the 
most commonly cited benefits of assessment centers. The first 


is by Ginsburg and Silverman [25] who described the design and 
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implementation of an assessment program for hospital personnel. 


They wrote as follows: 


Compared to other forms of personnel appraisal, this method 
1s seen to be more effective because all assessees: (1) 
have an equal opportunity to display their talents (2) 

are seen under similar conditions in relevant situations 
designed to bring out the particular skills and abilities 
needed for the position or positions for which they are 
being considered, and (3) are evaluated by a team of 
trained assessors, unbiased by past association, who are 
intimately familiar with both the position requirements 

and the institutional elimate . . . : 

The major contribution of the multiple assessment 
approach has been the use of situational tests or exercises 
. 0. . the application of situational techniques to 
assessment has reduced the amount of inferences which must 
be made from the more loosely structured paper and pencil 
techniques. They provide more positive answers to the 
question, "Given these traits, how is he likely to behave 
in a work situation?" 


Situational methods also offer the potential of adding 
greatly to the scope of human characteristics which can be 
evaluated. Although more expensive and time-consuming to 
administer than usual appraisal procedures, the need to 
find Ways of evaluating characteristics not coxered by the 
latter is sufficient to warrant extensive experimentation 
Wiutiemelatively elaborate techniques». . 


Ad] in all, the center does seem to be predictive of 
mamagerial ability. Furthermore, a vital part of the 
Identification and Development Center is the action which 
will be taken to increase individual effectiveness. Tailored 
developmental programs will replace shot-gun attempts at 
training individuals for increased responsibility. 


Expanding on the benefits of development, Slevin [59] 
wrote: 


The assessment center Can serve as a management develop- 
ment tool in tworways. First, the exercise of participating 
Na center is in itself a developmental experience. The 
individual gets practice in leadership skills, makes oral 
presentations, makes hypothetical management decisions and 
has them criticized, observes himself on a video tape 
replay of his performance and has other developmental 
experiences. Research has indicated that the video tape 
feedback alone has very promising potential for organizational 
development team building and other efforts (Weber [66)]). 
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Pi adawmeron, the whole process of the critical and intensive 
riod of self ansight that is experienced in a two to three 
day center is extremely constructive in providing the 
assessee with information on how to achieve positive change. 
He is provided with an opportunity to scrutinize his behavior 
in a critical manner not possible in on-the-job performance, 
and in a manner that might lead him to understand those 
behaviors that are most effective and least effective in 

his managerial capacity. 


Second, the assessment center is an excellent place for 
-the formulation of a developmental plan. Executives serving 
as assessors are currently operating at levels in the organ- 
1zation to which the assessee eventually aspires. They're 
a Sapo sPerenmtorevaltwate'’thestypes of experience and 
training the assessee needs to reach their level of attain- 
ment. At no other time in the assessee's experience with 
his organization will six upper management executives spend 
an intensive two to three hours discussing his strengths, 
weaknesses and developmental needs. At this point, in 
conjunction with the overall manpower needs of the organi- 
pa developmental plan may be engineered that is 
Specific to the needs of this assessee. This makes obsolete 
the conception of a standardized developmental approach for 
all management and provides the unique capacity of a specif- 
ically engineered developmental program for each person 
assessed. 


The third article is by Byham [11] who summarized 
assessment center advantages as follows: 


Reports have proved to be remarkably valid. Longitudinal 
studies of thousands of empYoyees assessed over the last few 
‚years indicate that this assessment method is much more 
accurate than traditional appraisal procedures, and these 
seem to be the reasons: 


*The exercises used are designed to bring out the specific 
skills and aptitudes needed in the position(s) for which 
eeeroup ofweandidates 1s beingeassessed. 


(Note: This facet is commonly referred to as "combating 
the Peter Principle; since the candidate is evaluated on 
requirements of the position he is aspiring to--not those he 
merzoheild in the past. When there is a large difference 
between these requirements--e.g., promotion from crafts to first- 
line management--this factor becomes very significant). 


*Since the exercises are standardized,assessors evaluate 


the candidates under relatively constant conditions and 
thus are able to make valid comparative judgments. 
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*The assessors usually do not know the candidates personally 
SO, being emotionally disengaged, they arë unbiased. 


*The assessors are shielded from the many interruptions of 
normal working conditions and can pay full attention to the 
candidates' behavior in the exercises. 


“The procedures focus their attention on the primary kinds 
of behavior they ought to observe in evaluating a promotion 
candidate. 


*They have been trained to observe and evaluate these kinds 
et behavior. 


“Byham further points-out-some-indirect benefits which 
Aide trom the center, e.g. 
a. Candidate training 
bD. Positive influence on morale and Jeb expectations 


c. Subtle improvement of candidates' understanding 
of and attitudes toward organizational goals and 
Perle res 


d. Assessor training (said to be by far the most 
valuable "fringe benefit"). 


As a final argument, Byhamtallies his survey of 


users of the centers: 


In a survey of the 20 companies that operated centers, 
I uncovered some 22 studies in all that showed assessment 
more effective than other approaches and only one that 
showed it exactly as effective as some other approaches. None 
showed it less effective. As I suggested before, these 
studies exhibit correlations between center predictions 
and achievement criteria such as advancement, salary grade, 
and performance ratings that range as high as .64. The 
companies appear satisfied that they are on the right track. 


Byham and Pentecost fl2] echo most of the preceding 


with the following additional observatıons: 


a. eo participation an an assessment Genter stimo tate: 
the candidates self- development through self-insight. 


b. Appropriate training and development programs can be 


preammeed and, if practieal,ejobs canmbe developed to match 
the particular abilities of available candidates 
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The report of a seminar conducted by the Foundation 
for Research on Human Behavior [112] illustrates the potential 
impact of face validity on group morale: 

It 1S important that the promotion of a given individual 
be not only the best selection but also that it appear to 
other candidates as a good selection. If a candidate is 
judged highly qualified mainly on the basis of tests, con- 
sultants' opinions, and other information relatively invisible 
to the candidate population, the advantages of promoting him 
have to be balanced against the possible disadvantages to the 
morale of the group. (However) if the candidates consider 
that the -evaiuatron ‘standards are clearly relevant to the 
job situation, they will tend to accept the results of the 

/ evaluations. 

Kraut and Scott [41] continue this theme: 

But face validity .. . is high for assessment programs. 
Raters in an assessment program generally come away feeling 
that it does indeed work. The exercises seem reasonable, 
and one can make discriminations which appear meaningful. 

They also report results of a study which they conducted 
involving 1086 subjects. The results showed that assessment 
center ratings correlated highly with two major criteria: 

(1) promotion to the second level of management and (2) demo- 
tions from the first level of management. Both of the criteria 
were chosen to minimize contamination. Since the assessment 
objective was selection for promotion to first level management, 
the authors contended that the criteria selected were minimally 
contaminated by knowledge of assessment center ratings. 

Kraut [40] cites studies by Bray and Grant [10], Wollowick 
andeMenamara [71] and Kraut and Scott [41]. In summary he 
States. 

These studies and many others lead one to conclude that 
assessment programs have validity in predicting those who 


will move ahead in an organization. Many of the studies have 
flaws, but there is a consistent pattern of apparent validity. 


52 








He also discusses a phenomenon reported by many studies: 
that the predictive validity of assessment center ratings 
improves as candidates are considered for promotions to second 
and third levels of management. He speculates on the reason 
hor this: 

It may be that the performance of first-line managers 
in this company is heavily dependent on technical skills 
Bere tae men caro into that level of management (first) 
and that the power of the assessment center shows up pri- 
“marily when they-are considered for higher levels of 
management. 

In reviewing a previous study by Hinrichs [32], Kraut 
points out a possible key to the advantage of assessment centers 
over traditional techniques. He says that observed interpersonal 
activity was a major contribution in the assessment, but not 
in traditional selection procedures. 

response to criticisms of “aeciding amman s Gare. 
on two days of observation,' he writes: "The assessment center 
May represent two days more of observation than existed before." 
course the Original criticism is fallacious since most assess- 
ment results augment rather than supplant performance appraisals 
and other sources of information used in promotion decisions. 

Dodd [18] reports a study from which he draws an 
interesting conclusion regarding assessment techniques and 
AS Teotypling: 

The present study offers some hope that, far from sup- 
Porting conformity or suppressing independence, tests vali- 
dated against assessment performance, when used to supplement 
Supervisory recognition, could break the hold of conformity 
on present selection procedures and open management ranks 
to those who value independence. 


The undisputed landmark in the field of assessment 


enter validation studies is the Management Progress udy 
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ENGI. No article on the subject can be considered inclusive 
woate paying homage to "The Study." Bray and Grant [10] offer 
the Most Gerinitcive articlevon ihis study and it is augmented 
by a series of articles amplifying on it and providing details 
of other AT&T validation studies. 

Bray and Campbe11 [9] describe a seudyson thesselecticn 
of salesmen which yielded a predictive validity of .51 for 
assessment center ratings correlated to job performance ratings 
obtained by special evaluation teans. 

Campbell and Bray [14] discuss criterion and contami- 
nation problems in validity studies. They cite two studies 
which attempted to overcome these. One was a Michigan Bell 
study which compared the first forty men asscssed and promoted 
With the last forty men promoted before the assessment program 
began. The other.study, done in the New England Company, compared 
candidates rated "acceptable" with thoserated "not acceptable." 
In both of these studies the ratings of the assessment center 
correlated highly with subsequent performance and were superior 
Bomtraditional selection procedures. The authors make a final 
mportames point that predictiwe validity alone is not sufficient 
but that acceptance by line management is also needed 1f a 
program is to be successful. 

Although Cohen and Jaffee [16] don't offer any advan- 
tages not previously mentioned, they do make threc predictions 
for the future of assessment centers in government applications: 
(1) that development will become a more important objective 


than selection or internal promotion; (2) that future programs 
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will utilize a greater number of new exercises simulating 
management situations in government, and (3) that the number 
of assessment centers in government will increase, as will 
interest in them. 

Miller [50] discusses the nature of potential and 
intimates that a single job may be an inadequate measure of 
a man's full potential: 

The so@emitted@poventialwei@an individual is not a single 
ability, but a combination of abilities to perform effectively 
oa a number of specific jobs. 

Slevin [59] writes to the same problem and points out the 
inadequacy of traditional personnel procedures: 

Thousands of psychological tests have been developed 
As pecar evalwlation’ of personality traits and mental 
Sera, but they tend to be rifle shot approaches that 
evaluate some narrow aspect of executive potential. 


E OCS mOn to point out that: 


Mimeontrast, themassessmemt center provides a“broad band" 
approach to the evaluation of executive potential. 


mother pun tementmonedmtrequently in the literature 
1s the effect of job assignment on an individual's career. 
Miller [50] describes it thus: 

Mitewecompaly is apt to place its high potential personnel 
ore challengine jobs, whereas low or no-potential per- 
sonnel will be left where they are or reassigned to less 
Eeapensile positrons. Thus they are offered little or no 
oer tunity towdasplay their qualifications for promotion. 

Mass it such assignment is based on valid selection procedures, 
a good man may be placed in a dead-end job and leit to stagnate. 
Miller refers to this hopeless situation as the "Darwinian 


' and points out the hope 


eheory of natural selection,' 
offered by assessment techniques. Since all candidates compete 


on an equal basis regardless of current assignment, the "have-nots" 


are not handicapped. 
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Pemerleau [53] makes this same point, but carrics it 
further. He points out the "image effect'' whereby managers tend 
to reward subordinates who act like them. This can utlimately 
lead to disaster, since: 


Selecting an individual primarily because he has no 
apparent weaknesses or because his background and attitudes 
correspond to incumbent executives often leads to the 
Selection of mediocrities who have no perceptible strengths. 


(Per amore detailed treatment"of this topic, See Senger [56] )® 
Pomerleau concludes that the assessment center is the solution 
m biased supervisors’ ratings Since it: 


. . . ensures that the individual is observed and evaluated 
by a team of trained evaluators unbiased by previous associ- 
ations, who are familiar with the skills to be measured, 
Meer sonal characteristics desired, and the environment 

in which the work is performed. 


A major advantage of assessment centers is the relative 
mmesedom from charges of bias in selection practices. Pomerleau 
MS” eopressessethe situation thus: 


Insofar as the assessment center technique is strictly 
job-related, employing exercises which seek to maximize 
Sbalectivaty, and further, since te concept Mas never been 
challenged by EEO (Equal Employment Opportunity) advocates, 
promoters of the concept may be encouraged. 


ties point is amplified inan article appearing in the 
muewst 25, 1971 issue of Business Week: 


Br. Cabot Jaffee thinks that, samcewthe assessment 
method is a job sample, it may be a way to avoid charges 
of discrimination that sometimes crop up over paper and 
pencil tests. He cites a Labor Department project 
Where he put a group of black and Indian trainees through 
conventional written tests and then through an assessment 
center. The conventional tests showed promotional potential 
only 16% of them= Assessmentetound it in 50% of them: 
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Peeamse Of this built-in “anti-bias" factory the assess- 
ment technique may get a boost from a recent Supreme Court 
ruling. Last March, the court ruled that, when a company 
uses personnel tests or degree requirements to screen candi.- 
@eees tor a job or for a promotion, it must be able to 
show the connections between its screening method and the 
Hebe" The”"court should be satisfied with assessment, since 
1t's based entirely on the job. 


Jaffee, ct., al. [360] address this subject indepth 
as does Dunnette [109]. A slightly different aspect of this 
general topic is covered by Bray [8] who writes on opportunities 
for women. He concludes that: "The assessment center can go 
a long way . . . toward opening up greater opportunities for 
the female employee and bringing about fuller utilization in 
Pnesshortest possible time." 
Despite the sources previously cited, there are some 
Caveats in the field of EEO which must be considered. Bray and 
Moses [76], in a review of the literature from 1968 through 
1970 offer these findings and conclusions: 
. . . Considerable attention was devoted to developing 
procedures which would result in similar test performance 
among racially mixed groups. Generally, none of these 
approaches have proved to be successful. The attempts to 
assess and compensate for bias in testing procedures have 
considered the following aspects: developing 'culture-fair" 
tests; using nonverbal tests; providing special test 
Somer twens; providing special test training 
The culture-fair fad appears over. One factor hastening 
its demise was the finding that nonverbal tests did not 
create additional fairness for disadvantaged group members. 
Nonverbal tests were initially viewed as approximations 
of culture-fair tests. Yet one of the few findings that 
consistently emerges in several studies is that nonverbal 
ects enhance differences. 
Howard [34] provides an excellent conclusion to this 


Section on assessment center advantages with the following 


summary of possible benefits: 





Beep with the Criterion Problem--Installation of assess- 
Meme procedures may force better job analyses and identifi- 
Bro Or the important criteria for success on a job. 
such a rigorous process has been aptly described in con- 
nection with an analysis of the job of foreman. . + Another 
way assessment centers may help with the criterion problem 
is by training assessors to evaluate more accurately the 
performance, behavior, and potential of others. Assessors 
have been shown to have greater agreement in ratings of 
feetercnt assessee traits. . ., but it has not yet been 
demonstrated that assessors will experience a transfer of 
training in rating subordinates under the unstandardized 
conditionsof the normal work experience. 


E EN Asas ors merne st bf.assessor tramite have 
Beewmeclained not only in the form of a partial solution of 
the criterion problem but through (a) improvement in inter- 
Viewing skills, (b) broadening of observation skills, 

(c) increased appreciation of group dynamics and leadership 
styles, (d) new insights into behavior, (e) strengthening 
of management skills through working with simulations, and 
(£) broadening one's repertoire of responses to problems. 
No well-designed training studies have validated these 
muamuses, however; as has been pointed out previously. 
firms do considerably more management research on selection 
ian On training and development . 


Development of Assessees--Since many exercises, like 
the in-basket and oral presentations, were used formerly 
as training exercises, many assume they serve such purposes 
in assessment centers, even without immediate feedback of 
meowlts ) .. Again, evidence supporting this training 
benefit has not been convincingly provided. Claims of 
increased self-insight. . have not been evaluated with 
Aretests Or control groups, and statements by™assessees 
that they felt the program was informative and useful for 
Be development, . . Cannot be accepted as firm empirical 
demonstrations of the developmental value of assessment 
genters. 


Minority Group Selection--Recent government interest 
Ice fairness of selection tests for minorities has 
Smee cad that selection procedures must be job related, 
and the simulated aspects of assessment centers do have 
faee Validity in this respect. One study bearing on this 
problem at AT&T. . . demonstrated that there were highly 
significant correlations between performance in a one day 
approximation of the company's longer Personnel Assessment 
aran and performance in the latter, regardless of race 
MES. The research design in this study more closely 
resembled alternate-form test reliability than predictive 
validity, however; thus the minority group fairness 
@uestion for assessment centers is not yet sufficiently 
answered. 
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Pacepvalidity--The simulation exercises in particular 
have high face validity, and the whole process has been 
claimed to be received favorably by managers, especially 
those who may be mistrustful of tests. . . . Some assessee 
ames tronnalres have also indicated that the majority consider 
the procedures useful and objective 


Attitude Changes--1t has sometimes been claimed that 
assessees may change their attitudes in the direction of a 
clearer understanding of some of the problems facing the 
managers and the necessity for making some unpopular 
decisions. . , but so far the evidence is anecdotal. 

In summary, the bonus benefits . . . of assessment 


centers sound promising but are largely without research 
support. 


TED advantages (Criticism of the concept) 

Hinrichs [32] reports a study, involving 1 m s5, Where i 
predictions of management potential obtained from a two day 
assessment program were compared with predictions obtained by 
wa dai tional technique. The traditional technique consisted 
of decisions made by two experienced managers who were provided 
With the Ss personnel records. Although the managers were 
encouraged to contact the Ss supervisors by telephone for addi- 
tional information, only a few such calls were made. Results 
of the study showed that the managers' predictions: " 
were as highly correlated with the assessment center data as 
were overall ratings from the two-day program, except for 
matings dealing with interpersonal behavior." The point of 
Hinrichs' study--and a criticism of it were made by Howard [34]: 

Ate tehs argued from this study that the™expensive two 
day assessment program may be unnecessary when MU Chek eee 
same information can be obtained so much easier. Inter- 
personal relationships seemed to be a relatively untapped 
area in the traditional system, however. Dunnette . . . 
fete. . . also disputed Ifinrichs' conclusions in that his 


correlation of .46 still left nearly 80 percent of the 
assessment ratings' variance unaccounted for. 
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To this writer, the real test would seem to be how 
vel: pemsonnel necomds or performance appraisals vs. 
assessment ratings can predict future management perfor- 
Nantes mot how well personnel records can predict assess- 
ment ratings. 

Another critical article is that by Wilson and Tatge 
UI ineir main thesis parallels that of Hinrichs; e.¢.. 
Phat other instruments (paper and pencil tests, clinical 
and patterned interviews, and interpretation of performance 
records) can rival assessment center results at a much lower 
Bose stating that: 

In the writers' opinion, there is simply no available 
or published evidence to support claims of the significant 
bumeriority of the centers over traditional methods. 

Although the authors challenge many claims made for 
assessment centers, they fail to offer any evidence to support 
Nr challenges. Like Hinrichs, they conclude that the 
greatest benefit of assessment centers is for early identi- 
fication when there is no performance record to provide an 
alternative evaluation. 

Odiorne [98] comments on a study of the AT&T program 
done on a sample of 500 managers some of whom had not been 
assessed and some of whom had received various ratings at the 
ases ssment center. 

This doesn't prove validity since, as Bray and Campbell 
enana tae assessment center is rarely overruled, and that 
At proves . . . A high assessment center gating 
seems to be itself a criterion for success, not a predictor 
of success. 

Mie was a validaeniticismapof the program in an operas 
tional environment, but it completely ignores other studies 
which controlled for such contamination. Bray and Grant [10] 


describe one: 
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The Bell System's Management Progress Study (Bray, 1964) 
offers a unique opportunity to study the assessment process 

The uniqueness of the study from the standpoint of 
Studying the assessment process arises from several aspects 
of the study design: 


1. There is no contamination by the assessment results 
of the subsequent criterion data. Along with all other infor- 
Maon collected on the 422 subjects of tite study tne 
assessment data are being held in strict confidence. Thus 
the judgements of the assessment staff have had no influence 
on the careers of men being studied. 


The other assessment problem cited by Odiorne was 

the effect of a rating offailure on an employee's later perfor- 
mance. This is spoken to by Kraut and Scott [41]. They found 
that the lowest rated group of candidates had a mobility rate 
one-half to two-thirds that of the total and therefore concluded 
that a poor assessment will slow promotion but not necessarily 
deny it. 

E with the previous section, Howard [34] provides a 
Succinct and comprehensive summary of the field as follows: 


The Crown Prince or Princess. Those who do outstandingly 
well in assessment centers may find that they have become a 
crown prince or princess. Management may treat them so well 
that their future success becomes a self-fulfilling prophecy, 
the morale of those without royal status may decline, and 
ticavalldity coefficients tor Whe assessment center process 
may become inflated. No research has substantiated these 
Pasential ceronation effects, however. 


The "Kiss of Death". A candidate who does poorly at 
an assessment center may feel that he has been given the 
kiss of death as far as his future with the company is con- 
cerned. This could result in some undesirable attrition, 
Since the candidate may be quite competent in the job he is 
now performing. Research on turnover of assessees so far 
has been inconclusive. 


Stress. If a candidate gets the impression that his 
entire career is on the line based on a few days "on stage,’ 
the stress effects could be quite strong. It would seem 
important that the data from the proccdure not be made of 
the pass-fail variety or kept too long in an employce's 
file. On the other hand, defenders of the procedures reply 
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tnarseıneenstresseıs a typical part of a manager's job, 

a candidate should be stressed to see how he copes with 

it. It would still seem important to keep stress in the 
exercises with limits. 


The Nonnominee. The feeling that an individual may 
be part of the "out group" if he or she has not been selected 
to participate in the assessment process (which may become 
a status symbol) is another dimension of employee attitudes 
that needs to be empirically tested. 


The "Organization Man." Some have raised the issue of 
Whether or TOC a: s Smient comers may not proliferate the 
model of the conforming organization man and serve to 
eliminate the unusual or imaginative managers that are 
believed to be needed in the future. A study at Sohio 
Showed that assessments correlated negatively with con- 
formity for one small sample, however. . .. An IBM study 
indicated that supervisors may nominate those higher on 
conformity and lower on independence, but that the assess- 
mMerteproceavesıtseils does not select this type of individual 
. « « The organization man may be the other side of the 
nonnominee problem; the most able and not the least able 
may be denied access to the assessment center. The impli- 
Cation 1s that it is the nomination procedure and not the 
assessment procedure that creates the organization man 
syndrome. The supervisory nominations should perhaps be 
Supplemented by self-nominat®ons, peer nominations, personnel 
records, or assessment of everyone at a job level if numbers 
are not too large. 


Costs. Estimates of costs have ranged from the price 
of a few meals to $5,000 per candidate, exclusive of staff 
salary. . .. Installation costs are the highest, but to 
these must be added assessors', assessees', and psychol- 
ogists' time, travel, accommodations, and meals, plus 
materials, from rating sheets to videotapes. Various cost 
Saving devices might include completing all possible pro- 
cedures before arrival at the center, conducting exercises 
On company property over weekends, and combining small 
companies with Similar jobs an a multiple company center, 
perhaps in a synthetic validity paradigm. In the end, 
these costs must be weighed in the context of current 
selection ratios against the possible gains in selection 
and training in some kind of a utility model. 


In summary, the possible negative outcomes of assess- 
ment centers, have much the same status as the (benefits), 
they appear reasonable, but for the most part they lack 
Supporting data. | 
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o. Design Considerations 
This section will review those factors seen as relevant 
to the design of an assessment center. In this area, Finkle 
and Jones [85], the Foundation for Research on Human Behavior 
[112] and Olmstead, et. al. [122] are the most pertinent and 
comprehensive references. 
a. Dimensions to be Assessed 
The threesseurzees just cited cover this topic quite 
thoroughly. Miller [50] and Kraut [40] offer general comments 
on how to ascertain relevant job Rens tons. Kraut offers some 
Particularly cogent advice: 
Most assessment programs are not based on an empirical study 
Of the manager's role, as might besdone through careful job 
analysis or a critical incident study. At best, they are 
based largely on a review of the research literature and 
the judgment of executives in the organization as to what 
makes for an effective manager. At worst, they tend to 
be copies of programs in other companies. 
Helme et. al. [114] describe the development of 
a combat simulation, and Krumm et. al. [118] writes on a 
research project to explore tactical military decision making. 
Although these articles are quite narrow in scope, they both 
deal with designs of military simulations and should be of 
some value to an assessment program designer. 
b. Number and Character of Assessments 
Several sources in the field suggest that more 
than one assessment of an individual is needed during a career. 
The basis underlying such a concept is outlined in Hall's paper 


[30] “A Theoretical Model of Career Subidentity Development 


in Organizational Settings." Hall.defines career subidentity 
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Le.) .lCw ee pemson'’s conception of himself in his career 
neler sehneeeocsmom to explain that subidentity (occupational) 
selecti N . . mas not simply a matter of selecting a role 
būt alsossme of’ choosanggangaspess ofsone's self whåch»is 
potentially effective and highly valued." He underscores the 
importance of this decis#on by pointing out that: 

. the career subidentity grows at an enormously faster 
Patean ihe tire cana suo sequent years. . „the 
earlysyecarseundeoubted]ly=erepresent a critical period for 
keasninszandleavezaslastinsinpnint on thes person's atti- 
tudes and aspirations. The expectations and standards 
internalized at this time will probably be enduring. 

These comments indicate a need for accurate assess- 
Deiat tile pPOllemobeocelpatronaimsclection (career entry) . 
Ha eaen introduces the concept of career develop- 
ment which hemdedames assa" . . = spiraling cembánationaef 
career choice, subadentity growth and conmatment." «He 
emphasizes the difference between occupational choice and 
parecen chol cC, explaining it as follows: 
Oconto nal ehanice the ehoice ot amcareer role, is made 
but two or three times in most lifetimes, while career 
ERolee, any ChotCemmamen wall agirecteone” secareer develop- 
ment, Can occur continmally: 
He points out that: 
. meeI Se OCI Olmepost-ClCOL10onN Career development repre- 
SClimesscm@ecpamillre tron Ene uswa]! psychological concern for 
ocupacional- chonce tne MROcesSsS Of Choosing a career role 
CONPTuUent Wabhecharacthemistnes wf whe individual. 
These latter comments indicate that the need for assessment 
does not end once the occupational choice is made, but continues 


morens t aS a eult olf the career choices hich "car occur 


contimually." 
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Pomerleau [53] expresses the same basic concepts 
in a more pragmatic way: 


When we speak of an “identification system," we need to 
be reminded that it has two subsystems: (1) Early identi- 
ricseaon of hieh potential (EIHP) and (2) Identification 
of higher potential within the midmanager ranks. Funda- 
mentally, the first subsystem asks: What could the raw 
candidate, devoid of any meaningful performance record 
beyond academic achievements, do,given the opportunity to 
perform in challenging tasks? The second subsystem asks: 
What has the midmanager done so far to distinguish himself? 
Our source of data can be culled from job performance, 

meperrenee,- developmental-assignments, supervisor appraisal, 
etc., and then determine what kinds of plans could be 
Mapped out for him to realize the full measuwre.of his 
managerial potential. 


Slevin [59] has a slightly different view of the 
concept and is the first to specifically describe the "assess- 
ment center approach:" 


The assessment center is conducted in the usual fashion 
with an emphasis on both evaluation and the diagnosis of 
developmental needs. These two variables may then be fed 
into an overall manpower plan. This plan takes into account 
Meee i ronnen Cal demands on thé firm and the projections 
of future manpower needs in various areas. Based on his 
own competence and what his developmental needs are, an 
individual will be plugged into two plans: a promotion 
program that is specifically constructed for the individual, 
and a management development program. The promotion pro- 
gram specifies a career path that the individual is likely 
torio. 1n the organization. The management development 
program focuses on the difference between the individual's 
Bias nitscompetence anasruture needs as defined by his 
Pemeetive career path. Using both the promotion and 
management development programs, the organization can 
implement an effective and farsighted manpower plan. 


The assessment center is probably the best example of 
a successful "broad band" approach. Because it is able to 
fill both organizational appraisal and developmental needs, 
the assessment center is likely to flourish as we prepare 
our Organizations for the 1980's. 
Pieler i530) Makes One of the most explicit state- 


ments on the subject: 


65 








Moreover it iS an error to assume that an individual's 
HARI Lor promotion to the next higher level or to jobs 
several levels higher remains constant. Job requirements 
change, organizational goals and demands shift, and an 
individual's personal qualifications are likely to change 
with time; hence his potential for promotion to various 
ISI in time. Therefore potential must be period- 
ically reassessed. 


In conclusion, Hinrichs [32] summarizes the diverse 
objectives of an assessment program as follows: 

: one hopes the assessment program will be able to 
MIE pronogablc. people ™earlmer in their careers, that - 
mews! Neip to clarity some or the skills important in 
promotion, and that it will perhaps identify some people 
who should be promoted but who might, under normal circum- 
dances, be overlooked. Hopefully, also, the program will 
fulfill, to some extent, a personal development function 
Pee providing practice in group situations, individualized 
feedback regarding observed strengths and weaknesses, and 
greater understanding regarding the caliber of the competi- 
tion participants are up against. 

c. Uses of Results 

The output or result of the assessment process can 
ime utilized in several different ways. This is not to imply 
Met a Single program could meet all of the diverse objectives, 
IS ome tot the objectives are in conflict, e.g., selection 
nally calls for stable variables to be measured, while 
unstable variables are the focus of developmental programs. 
because of this, 1t is important that objectives (primary and 
Secondary) be specifically delineated before the program is 
designed, to ensure highest efficiency. 

Finkle and Jones [85] cover this topic most 
mroroughly in Chapter 15 of their book. They identify the 
following uses: 

Selection decisions 


Ha ame 
Eir iny 
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Transfer and placement 
Organization requirements 
Individual requirements 


Management development 

Organizational planning 

Counseling and consulting 

Additional uses suggested by Culclasure [108], are 
necer motivational prediction and selection. Although his 
wdy failed to develop predictors, he wrote: 


Available evidence, however, suggested that a career moti- 
fomeloOne SCEeecn lm mroceaumes COudia be developed . . „ Such 

a procedure would employ (a) those portions of the Strong 

Vocational Interest Blank (SVIB) which have been shown to 

have moderate validity (0.25) for predicting Naval officer 
retention when scored with the Navy Officer Key, and (b) 

a revised version of the Importance-Possibility Scale. 


Helme, et. al. [114] also address military assess- 
Pent applications as follows: 


Barly Identification of otficer leaders and development 
of officer leadership from cadet training through company 
and field grade assignments are of major concern in the 
Management of the Army's manpower resources. The Behavior 
and Systems Research Laboratory (BESRL) conducts research 
Bor provide scientific means of identifying individuals with 
mood leaderShugs Potential for sod fauccr simal nin gy» Selectie 
Officers for commissioning, and evaluating their performance. 
mene Ppaesent series sf publicatiens recordsathe conduet of 
wong terim experimental program to improve the basis for 
selecting ana developing ofticer leaders in accordance with 
Mear capability to meet dit.erine leadership requirements. 
Differential prediction and evaluation have become dominant 
Ebjeetives in the effort to channel officers into appropri- 
m assignments and develop their potential so as to make 
Dest use of their abilities. 


As mentioned previously, a single assessment 
sopram is realistically confined to one or two objectives. 
Mees iS illustrated by the fact that studies of operational 
mesessment centers rarely address more than two objectives. 


The following examples are offered. 
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Miller [50] cites selection and identification; 
Pomerleau [53] stresses identification with a secondary function 
of development; Slevin [59] says essentially the same thing, 
Stating: "No two problems are more crucial in today's changing 
organizations than effective management appraisal and develop- 
ment." Kraut and Scott [41] describe a combination of assess- 
ment and development: 

The assessment program consists of two and one half days of 
assessment activity patterned after the AT&T model, plus two 
days Of murely developmentede actiwaty . . . The development 
portion includes Wectures, seminars, and classroom discussions, 
as well as a feedback interview. 

Jelks [37] makes an important point regarding the 
use of assessment center information: 

The program is not designed as a final judgment factor in 
the selection of management personnel, nor does it replace 
the supervisory appraisal of current job performance. It 
Simply makes one more piece ot information available that 
can be fitted into a record of employee performance. 

Kraut [40] makes the point just stated, and another, 
equally important one: "The data should . . . be discarded for 
any decision making purpose after two years or so." Interviews 
disclosed a similar policy at Pacific Bell Telephone with a 
figure of about three years mentioned as a maximum "life" of 
data for selection purposes. In contrast, Finkle and Jones [85] 
mere a period of "five to ten years, or longer." 

Menal ource in this section makes a prediction 
on the future uses of assessment centers. Bray and Moses write: 

Byham's . . . [11] . . . review of personnel research activ- 


ities w@mdicates a growing interest in developmental and 
placement strategies rather than on selection per Se. 
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isterdaeciding onethe objective(s) of an assess- 
Meme program, the designer next faces a myriad of decisions. 
The more basic of these will be arbitrarily lumped together 
wa considered in the next section. 

dem Basic Considerations 

Hiwice 12actors Gensidered basve to assessment progran 
design will be reviewed. These factors are: (1) "Signs vs. 
tes, "(2)" Stable vs sunssableswariables; and (3) Statistical 
ms. clinical combination of assessment results. 


(1) Signs vs. Samples. Bray and Moses [76] write: 


mone and Camppell . .. [67] . . . argue that tests and 
eber predictors should be used as samples rather than signs 
Of behavior: In other words, the measures we use to predict 
Should be measures of behavior. 


Howard [34] seconds this view by noting that: 


A unique contribution of assessment centers is the inclusion 
Of Situational tests in the assessment battery. The rationale 
behind using such exercises is that they simulate the type 

of work to which the candidate will be exposed and allow his 
performance to be observed under somewhat realistic conditions. 
memtrary to the aptitude test approach, samples, not signs 
meoeiavior are used for prediction. 


Wernimont and Campbell [67] contend that samples are superior 
to signs in the following ways: (a) less bias due to faking 
Myaneresponse sets; (b) less discrimination in testing because 
Samples are more relevant and fair than signs; (c) less invasion 
Memerrivacy With Samples. As to the relative effectiveness of 
ignis vs. samples," Bray and Grant [10] stated that: 

ine date epe@etcCdemake it apparent that the situational 

techniques (group exercises and In-Basket) used in the 

Emna rement Progress study produced, despite their complex- 

aties, reasonably reliable results and that they markedly 

influenced the judgments of the assessment staffs. The 

Peper-ana-pencil instruments (signs) had less influence 

on Staff evaluations generally, though they did influence 


them in many Specific ways. 
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Wollowick and McNamara [71] refer to the preceding source and 
Speculate as to the cost-effectiveness of simulations (samples): 


An important question frequently raised (Bray and Grant, 
1966) can also be partially answered from these (Wollowick 
and McNamara's) results. That is, can the assessment pro- 
cedure be justified in light of its additional cost and 

time compared to the use of paper-and-pencil tests alone? 

° . +. inclusion of the elements unique to the assessment 
Seller procedure . . . nearly doubles the criterion variance 
accounted for. This indicates that the assessment procedure 
makes a substantial unique contribution to the prediction 

of management success. 


(2) Stable vs. Unstable Variables. The considera- 
Pron given to this question in the design of the Sohio program 
WS described by Finkle and Jones [85]: 


7. the tenpueasis an tie program at Soo *"as on the 
development of information that might be expected to be 
reasonably stable and consistent over some period of time, 
perhaps five to ten years or longer. Studies have been 
made as to the degree to which measurements and judgments 
of abilities, interests, attitudes, and values of individuals 
tei tO Change over time. It seems appropriate, in establishing 
äl group ol basic abilities and characteristics, to make use 
er avaılable studies, general experience, and common sense 
to eliminate from consideration variables that are highly 
fuineraplc to change during a relatively short period of 
time. 


However, this approach clearly presupposes that the 
program will establish less information pertinent to training 
and developmental plans than might otherwise be possible. 
Immediately the question arises: Is it not possible to do 
both, namely evaluate some characteristics not likely ta 
change and others more likely to? The answer, of course, 


Is. DOE aperodenes could” De taken, but the orienta- 
tions are essentially in conflict. Our emphasis in Sohio 
was placed on stability. It was not planned to reschedule 


the elaborate assessment procedures of this sort in any 
Short period of time since it was felt that it was important 
to be as fully aware as possible of more permanent charac- 
teristics when making manpower decisions, particularly those 
of a long-range nature. 


SES At TS tLES vs. Clinical Combination of Assess- 
ment Results. Bray and Grant [10], in describing the AT&T 
Management Progress Study assessment program, make a rather 


one-sided statement: 
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ASSESSMENT procedures @lso contrast with psychometric 
ones in the way the resulting data are combined. Psychometric 
approaches depend on mathematical methods for accomplishing 
this purpose whereas assessment approaches combine the data 
judgmentally. 


Matt [61] 1s moresobjective on» this podnt and offers an excellent 


analysis of both sides of the question: 


We have argued that there are occasions when intuitive 
methods of making predictions, 1.e., "clinical" have their 
appropriate place. Statistical methods cannot be used 
where no prediction formula exists. But some personality 
assessors speak as if the clinical method is always to be 
pmeterred asemt enablés» the assessor to be flexible in his 
mee Or the data in a Way that is not possible»with statis- 
tical techniques; for example, the clinician can give weight 
memobVvi0us but rare anad@iWwonrepeatable factors in the subject's 
current situation which could not be validated empirically. 
Other advantages claimed for the clinical against the 
Seatistical approach are that it does not violate the essential 
naty of the subiiect"s personality, sand that 1t enables the 
mse of empathy and recipathy im making the predietions. 
(Actually these subjective clues could also be used as data 
by the statistician along with other more objective data.) 


Other assessors regard clinical techniques as only a 
last resort. A number of advantages can be quoted for 
Geatistical prediction over clinical, most of which boil 
Mew to the fact that the statistician has æ far Moreweffic- 
Ment memory and a larger attention span than the clinician; 
he can "remember" the relevant data at the appropriate time 
and combine them with other data in order to obtain optimal 
meichtings for future predications. 


And so we have, on the one hand, the efficient but rigid 
and Inman statistical prediction, and on the other, the 
ilexible and humane but ineffWwelent clinical. Which one is 
more useful in personality assessment? 


Me choice of Method will depend*en both the requirements 
and the over-all situation, including sometimes, public 
relations considerations. The final selection of assessment 
techniques is likely to be a mixture of both subjective 
Mimonvective, DUL the circumstances that will favor one or 
we othe at any stage are rather vague, and the choice is 
usually made on subjective grounds, although it, too, could 
be Made Ci the basis Of obiective, empirical investigation. 
Pie cgeieral, objective methods are to be preferred as far as 
possible as they maximize accuracy, but practical consider- 
ations of economy, convenience, and the limitations of the 
Situation, dictate the wholesale use of subjective methods 
in personality assessment. These subjective methods may 
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have high validity under favorable circumstances, and where 
the assessors are familiar with the criterion situation, 
clinical judgments may actually be more accurate than any 
Objective methods are ®ver likely to be in predicting to 
Briterla. 


Wollowick and McNamara [71] report an interesting finding from 
mheir study: 


Milessubjectively derived combination of tliesvariables (owerall 
taring) correlated .37 with the criterion, while the statis- 
tical combination gave a multiple of .62. This suggests that 
instead of deriving an (overall rating) by subjective mean(s), 
it might be done more profitably by a statistical procedure. 
This should greatly increase the predictiveness of the 
program. 


Meehl [96] makes a very positive statement on 
Ehe issue as he summarizes a number of studies on this subject: 


In spite of the defects and ambiguities present, let me 
maenasiZe othe brute tact that owe have here, depending upon 
one's standards for admission as relevant, from 16 to 20 
Studies involving a comparison of clinical and actuarial 
m nods, in all but one of which the predictions made 
memvarially “were either approximately equal or superior to 
ao Cline tan. Italics are Meehl's.) Further 
investigation is in order to eliminate the defects mentioned, 
and to establish the classes of situations in which each 
meenod 1S more efficient. | 













Lest Meehl's position be misconstrued by quoting him out of 
mentext, his “thesis in a nutshell" is quoted: 


Tiere is not convincing reason to assume that explicitly 
formalized mathematical rules and the clinician's creativity 
mececQualiy Suited for any given kind of task, or that their 
comparative effectiveness is the same for different tasks. 
murrent Clinical practice should be Mich more critically 
examined with this in mind than it has been. 


Ome final viewpoint on this" subject “is offered. 
Howard [34] cites yet another study, and makes some suggestions 
for future assessment programs: 
Again we find a demonstration of the usefulness of 
clinical measurement but the superiority of mechanical 


Sempination ol data . . . (Sawyer [56]). Although one 
Matas Cannot be generalized to all assessment centers, 
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it certainly appears advisable for other centers to research 
the hypothesis that mechanical combination of data may improve 
Preurctions even more. Should this prove true, once research 
costs were recovered, the unit cost savings of reducing 
assessors’ time could be substantial. 


e. Instruments and Techniques 

The designer of an assessment program faces a 
formidable task in the selection of instruments and techniques 
which will best serve the objectives of the program. Compounding 
hes task asethesplethora ofveemmercraleanstruments vying for his 
approval. 

Mibrighte et. “alle @i7s)|) offer a relatively brief but 
Eenprenensive coverage of “the use of psychological tests in 
industry.' Bray and Grant [10] provide an overview more relevant 
to assessment programs with their listing of the instruments 
included in the Management Progress Study: 


Me methods uscdstor collesetimg infommatlon: ome the 
persone) Gharacteamistics of thespartMelpemts ame repHes ena - 
tive of these used @mener@lly in@@sse@sstiemt activities. A 
Bistines of the techniWes*KNaith a brief description of each 
follows: 


Interview. A two-hour interview with each man directed 
at obtaining insights into his personal development up to 
Behatstrııe, Work objectives, atífltudes toward the Bell Systen, 
social values, scope of interests, interpersonal relation- 
Ships: idiosyncrasies, otc.. 


In-Basket. A set of materials which a telephone company 
manager might expect to find in his in-basket. The items, 
25 altogether, range from telephone messages to detailed 
reports. In addition, examinee was furnished with such 
necessary materials as a copy of the union contract, organi- 
zation chart, and stationery. He was given three hours in 
which to review the materials and take appropriate action 
on each item (by writing letters, memos, and notes to himself). 
Following completion of the "basket'' he was interviewed con- 
cerning his approach to the task, his reasons for taking 
bhewactions indicated, and his views of his superiors, 
peers, and subordinates (as inferred from the materials). 





Manufacturing Problem (made available by Joln Hemph111 
ot the Educational Testing Service). A small-business 


game wherein the participants assumed the roles of partners 
in an enterprise manufacturing toys for the Christmas trade. 
ines participants were required to buy parts and sell finished 
products under varying market conditions, to maintain inven- 
tories, and to manufacture the toys. 


Group Discussion. Also a leaderless group situation, 
focused around a management personnel function. Participants 
Were instructed to assume the roles of managers, each having 
a foreman reporting to him considered capable of promotion. 
Participants were required to discuss the merits and 


liabilities of their hypothetical foremen and to reach a group 
E cision regarding Teir relative promotabilities. 


Projectives. (a) Rotter Incomplete Sentences Blank 
(published by the Psychological Corporation). (b) Bell 
Incomplete Sentences Test (by Walter Katkovsky and Vaughn 
randall with the advice and assistance of Julian Rotter). 
(c) Thematic Apperception Test (published by Harvard 
University Press). Six of the cards from this test were 
administered. 


Papermend-mencil tests and Questionnaires. (a) School 
wC Coll ece Ability test, form 1 (published by the Coopera- 
meve Test Division of the Educational Testing Service). 
Ponually since 1956 the Personnel Research Section of ATT 
mas developed, following the Educational Testing Service 
format, its own version of this test. (d) Edwards Personal 
Preterence Schedule (published by the Psychological Corpor- 
ation). (c) The Guilford-Martin Inventory of Factors GAMIN 
(published by the Sheridan Supply Company). (£) Opinion 
MMestionnaire, Form B. Unpublished, this questionnaire, 
made available to the Bell System by Irving Sarnoff of New 
York University, is designed to reflect a person's attitudes 
toward making money and advancing himself. 


Miscellaneous. (a) Personal history questionnairc. 
we nort autobiographical essay. (c) Q sort (70 items, 
Belf-descriptive). 


Another “werviéw can be found 1n Finkle and Jones 


[85]. They provide the following informative table: 
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Figures? 


Suggested Factors Indicating the Value of a Test or Procedure 








IMPORTANCE Gi Mie BETTER LOWER THE BETTER 
eee critical Relevance Unacceptability 
importance Nolte protesisa comal to assessees 
confidence) 
Of considerable Uniqueness Time to administer 
importance Acceptability to and process 
management Cost in time of 
Aceepeability to management 
assessees and professional 


personnel 
Cost to purchase 
Cost towdemel op 
or improve 


Of some importance Change of pace Restriction on 
research 


Bource: Finkle and Jones [85] 


They continue with some additional suggestions to consider in 
the selection or design of tests: 
Mental Ability Tests 


Beyond the general criteria of relevanec, wniqueness, 
acceptability, and the others already considered, it is 
MN choosing mental “ability tests for the assess- 
ment program to seek ones for which appropriate norms are 
available. Here we mean tables or statistics showing what 
mts be considered typical, or relatively high or low, 
scores on the test for persons similar, in whatever quality 
the test measures, to those to be assessed. Preterably, 
mests can be used for which norms on managers or executives 
IM business are available. Such information can usually 
be obtained through test publishers or often by cooperative 
arrangement with business organizations that have used the 
Mater, as sufficient experience is gained, the com- 
pany with the new assessment program can establish its own 
norms and standards based on scores of its own personnel. 


Erojecetive Personality Tests 
In projective tests the psychologist is considered to 
Beast of the test. Therefore, the choice and make-up 


TO yective tests should, to a considerable extent, be 
Min the hands of the clinical psychologist who uses 
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them. However, it may be well, in choosing the clinical 
psychologist who will particpate in the program, to consider 
Pier etnie projective tests he prefers to use will prove 
generally acceptable to assessees and managers in the organi- 
ton and will be acceptable to other clinical psychologists 
who may help him or replace him in the program. We doubt, 
for example, that ink-blot or word-association tests are as 
Meceptaple in business as thematic or sentence-completion 
tests. And if the sentence-completion procedure is to be 
med (particularly if one is to be tailor-made for the 

mM ogram), it becomes important to have some assurance that 

it is acceptable to psychologists other than the one who 
designed it. 


mietiple-Choice Personality and Interest Tests 


Siero Of testing has been the principal focus of 
negative criticism of management level testing in business. 
Accusations have ranged from unfairness, through perpetuation 
erscontormity, to simple ridiculousness. Anyone considering 
the use of such procedures should obtain good professional 
menisce! Since much of the criticism, though so frequently 
and sensationally expressed as to alienate most professionals, 
mas some. basis in fact. 


Two characteristics of such tests probably account for 
Mo the criticism leveled at them. First, they rather 
Bevtously can be faked. This by no means necessarily 
MESES in a distortion that is favorable to the "faker." 
memcdoes, however, reduce his confidence in the interpretation 
of the test and therefore in the fairness and value of the 
Brent process. Furthermore, it may prevent the effec- 
tive use of the scoring and weighting system originally 
Beveloped for the test without, at least, a follow-up study 
fo examine the effects of attempted faking on the scoring 
System that was, in most instances, developed from answers 
given by persons taking the test experimentally or for 
counseling purposes. Second, the basis of development and 
moring Of such tests is actuarial rather than clinical. 
This means that the choice of items and the scoring system 
are ultimately based on statistical relationships between 
actual answers to the questions and one or more other 
measures such as judgments of performance or personality 
Constructs. This relationship is accepted as an empirical 
fact and efforts at making a rational interpretation of 
individual item relationships are usually not attempted 
or considered relevant. Consequently, anyone judging the 
contribution of the test by reading and thinking through 
the answers, or by examining the weights given the answers, 
may simply miss the mark in evaluating the test. 


or 








ese Characteristics of multiple-choice personality 
PSSS DONT to the importance of a careful examination of 
their value under conditions of actual use as a means of 
weighting their contribution against their dubious accept- 
AA In selecting tests a review of the questions for 
Study can give some clue as to the reaction that can be 
anticipated by test-takers. 


Interviewing 


Interviewing can presumably be called a type of assess- 
ment procedure, but there seems to be nearly as many ways 
of interviewing as there are interviewers. As discussed 
earlier, arguments can be made for having all interviews 
inan assessment program conducted by psychologists or 
other specially trained persons. Other arguments support 
the value of having managers conduct the interviews within 
the frameworks of some assessment programs. The Sohio 
program adopts the latter approach and in fact calls for 
a constantly changing (from program to program) group of 
managers to conduct interviews with a minimum of special 
ei eruction or training. 


Clearly, the value of interviewing under such conditions 
comes from maximizing the already developed habits of the 
Managers rather than in attempting to change or "perfect" 
their skills in accordance with some predetermined style or 
Pareern of interviewing. In a brief "orientation" meeting 
prior to their conducting interviews, the managers are 
asked to review personal history forms of the four individuals 
each of them will interview. It is pointed out that these 
interviews differ from ones they have held in the past in 
that this time they have no job to describe or "sell." They 
are encouraged to look for answers in the personal history 
form that can be used to encourage the interviewee to 
elaborate on why he did a certain thing such as choose a 
mege. a major in college, a company to work for, or a 
change of company. They are also asked to point out to 
Meter viewee their unique role as his representative to 
the assessment committee: How would the interviewee like 
to be represented? What does he feel he is best Suited 
to do in the company and why does he feel he is so suited? 
The managers are cautioned that some interviewees may 
eeepc to turn questions back on them and it is suggested 
Mat ciey retrain from describing their own backgrounds 
and activities until the formal interview it over. 


A few sample interview reports from past programs (edited) 
e presented and it is stressed that their own reports should 
contain two things: information about the interviewee and 
their judgment about him from the interview exposure. No 
mercer 1S recommended in organizing the report. In fact, 
Mesale interview report 1S written chronologically from 
childhood while the other begins with the first Sohio job 
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and uses earlier experience information to support and 
elaborate on opinions and judgments about professed skills 
and interests. The managers are offered tape recorders if 
they do not wish to write out their interview reports in 
M This approach, though it is only slightly struc- 
offers little or no advicë on style, has resulted 
in most instances in the production of effective 1-1/2 io 
2-1/2 page typewritten reports. 


Situational Exercises 


So far as we know, situational exercises have to be 
developed or borrowed rather than ordered as published 
metsHons throuehn a catalogue. Such procedures can provide 
Bither”individual-or groupractivity. The overriding objec- 
tive of the approach is to produce activity based on business- 
like matters which can be reliably judged by observers or 
reviewers. It is presumed, though sometimes given secondary 
attention, that the judgments (or observations) produced 
can be weighed compatibly with other information to produce 
Tene ruil Combined judgments helpful in future selection, 
placement, or developmental decisions. A critical aspect 
Breiter ıonal exercise, therefore, is the realism (in 
ward to business problems) in the eyes of those partici- 
pating in it and in the eyes of those judging it. Accordingly 
e ther individual Or group exercises, the task provided 
Should bring the assessee as close as possible to realistic 
problem-solving and should require as little 'role-piayinge' 
as possible. Observers should be able to hear or observe 
wounen to give meaningful interpretation to what occurs 
rather than to have to infer what a person is thinking or 
feeling. Some thought should probably be given to estab- 
Menang a variety of tasks so that persons with varied 
ateo rounds, skills, and characteristics find opportunity 
Bemaemonstrate their talents and styles. For example. 
while one group exercise might call for quick and con- 
mineing argumentation based on little specific content, 
another might call for some quantitative analysis as a 
Prereauisite to discussion, while still another might 
feature sensitivity to the feelings of others as a basis 
Map propr1ate contributzon. The exercises might also 
vary in the amount of preparation allowed in order to 
contrast planned and rehearsed behavior .with impromptu 
behavior. 


IN e rclses. the determination and explanation of 
exactly what the observer iS to note and report is quite 
portant. In individual exercises, the observer may examine 
the results of the exercise (as in the in-basket exercise); 
he may observe individual performance (as in the presentation 
NO Mi talks); or he may actually question the assesse®ee 
Beta why he took certain actions (again, as in the in-basket 
exercise). In some individual exercises, observations may 
mot be made, but: rather scores (or counts) may be obtained 
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by formula and reported to the assessment committee (as 
in a stock market exercise developed and experimented 
meth at Sohio). 


For group exercises, observers may have specific 
assignments to concentrate on the performance of one or two 
of a group of individuals who are engaged in a leaderless 
group discussion (such as reviewing a case report) or in 
a Simulated work activity (such as a marketing-manufacturing 
problem). Here the observer may be given considerable or 
little structure ranging from using a checklist to mark 
what he sees to simply writing notes about what he sees. 
Fuchenotes can then provide him with the basis for preparing 
A E TON O mt Omori tetling' the assessment committee 
what "his" assessees did during the exercise. 


An observer could, of course, be asked simply to com- 
plete a series of rating scales Similar to or even identical 
me the Skt and style Variables to be used by the assessment 
Sommattee. However, we feel that this approach has two 
disadvantages. First, the other members of the assessment 
committee, who concentrated on other assessees or who did 
mot observe the excrcise at all, will have no way of joining 
in the interpretation and judgment of the performance of 
fhe individual being reported on; they must simply accept, 
Enot mae cept the gjudomentsmot the reporting oObseamer. 
second the Observer may, Oy Tating the assessee on, the 
meriables solely £rom observatiémes» in one exercise, over- 
commit himself on each rating prior to hearing evidence 
som the other exercises and procedures. To offset these 
messiple probtens, we suggest that observers be given a 
checklist or structure that produces broad coverage and 
descriptions without calling ror judgments on the basic 
program variables. A highly detailed checklist seems 
advisable only when the observers can be well trained in 
1ts use--generally when management observers are expected 
pom eme libras” star contrıibutors through several programs. 


Though somewhat less important than other considerations, 
some method of ready identification by observers of all 
participants at a group exercise fosters better association 
of remarks with those making them. We have adopted the use 
Of colored vests--other approaches could probably serve as 
well. Observers otherwise will have no trouble identifying 
mie persons they are assigned to concentrate on--but they 
nieht have difficulty in identifying exactly who their 
assigned assessees are talking to or with in the group-- 
particularly if a couple of the “other' assessees have their 
Mecks to the observer. By having each participant in the 
Moup exercise clearly identified (from all directions) by 
moor, the observers can merely note interactions by these 
mbols, and when their reports are later typed, the typists 
Man substitute the correct names for the colors. 
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oee inal DIt ol advice for the designer is offered 


by Hardesty and Jones [31], e.g.: 

In addition to the variety of assessment approaches used 
sche program, the designers purposely attempted to build- 
in duplication when possible and to strive for some overlap 
among and within the assessment approaches in order to 
increase the confidence they could have in the assessment 
information. 


The remainder of this section will continue to 
scomsider specificuinstnuments and techniques individually. 

Come enOrcemmc Personality Tests. "The best 
Meee receround On this subject is "Contributions of projective 
m-chniques to assessment of management potential," by Grant et. 
IZ. the authors conclude that " .. . projective reports 
Mearticularly influenced the assessment staff in rating such 
characteristics as work motivation, passivity, and dependency. 

AE ENS Mb Taper tinent articles 
have been written on interviews. Grant and Bray's [26] "Con- 
mreibutiems of the interview to assessment of management potential," 
one Ot them. They found that: “Analyses of the data 
meearly indicate that information from the interview reports 
contributes to assessment center evaluations." 

“The validity of personnel interview" by 
Mis cita [23] 1s a brief but Worthwhile article. His conclusion 
Bates that: . . . "the ordinary personnel interview is not 
necessarily and invariably invalid, but rather that its validity 
eebe at least equal to, if not greater than, the validity 
BL tests." 
Pr gene ero rusa study to determine the 


effectiveness of interview information as a predictor of future 
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Job performance. He concluded that: 


Psychologists differentiate individuals through interviews 


on two general dimensions: Personal Relations Effectiveness 
SiGe iiteiteceval “Functioning... . . 
and 


Psychologists can predict the potential job performance of 
individuals on the basis of information and impressions 
obtained in an interview. The interview has some validity 
fOr the assessment of higher-level personnel. 


(3) Simulations/Situational Tests. This category 

can be subdivided into.group and individual tests. Those articles 
dealing with the entire field will be reviewed first, followed 
Dy those on group exercises. Individual tests will be covered 
ast. 

Pinkie andi Joneses] again offeraaps excellent 
ereatment which 1s enhanced by the inclusien safsadven tages 
and disadvantages of situational techniques: 


Mts tapa ch GolsilStS Or Onc Or More Standard exercises 
eepeercayto bring Out ability and personality characteristics 
directly and more obviously related to successful performance 
in supervisory, administrative, and managerial responsibil- 
ities. These exercises may consist of individual tasks, such 
as writing recommendations about a variety of letters, reports, 
Mmomerees., and SO On, typical of the contents of an executive's 
"in" basket. Or they may require elaborate group activity 
of six or eight candidates "running" a simulated company 
Or “Serving on a committee.™ To some degree how well the 
Situations are handled is informative: for example, the best 
recommendations on the in-basket items, higher profits in 
the company, 0. contro lMlane the committees There is, 
however, some question as to whether or not results on these 
situational exercises guarantee results in a real liss situ- 


ation. Much more important is the picture obtained of the 
Apes Skilltis, Styles, and personalities of the partici- 
eneee Such characteristics are noticed as: (1) desree of 


organization, (2) willingness or reluctance to make decisions, 
(3) willingness to take action, (4) clarity and impact in 
Soumunmeduetlons, (5) awareness and sensitivity to the thoughts 
and feelings of others (6) impact on group decision-making 
and (7) approach to problem definition and attack. 
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Ice re tatioón Of these situational activities may 
Wary Considerably from a rather intuitive, commonsense, or 
clinical judgment formed by either psychologists or manage- 
ment personnel as to the significance of certain answers or 
Herons to, On the the other hand, a rather empirically 
related analysis of the answers and actions as compared 
cien ua ements in terms of ratings, later actions or 
future consequences. In other words, these techniques may 
be approached on the basis of the standardized selection 
een les such as regular paper-and-pencil tests, supposedly 
supported by validation studies against other measures 
obtained after employment; or they may be judged more as 
MicmetintecdleanttOrmiacion 25 yudged by the clinical psycholo- 
St. : 


Advantages 


a. The problems and situations used with this approach are 
designed to bring out characteristics and abilities that 
ecieedrcet ly relateasana 1elevyant to success in the business 
world. 

meee these techniques involve some important areas, such as 
meterpersonal relations, administrative skills, and communi- 
OS a DA tratare not well covered by the other 
Beechniques. 

c. These techniques are readily adaptable to administration 
end use by business personne! as well as by psychologists. 
Ie ae scriprtions of the Styles and abilities of.a1ndi- 
viduals as assessed by these techniques are more readily 
understandable to management personnel making promotion 
Mess tons than are some of the descriptions written by 
Nsyenologists tollowing the clinical approach or scores 
Hebived trom the statistical approach. 


Disadvantages 


ie lhis approach can be relatively timezconsuming, depending 
bon the number ol exercises employed. Most exercises take 
from one to three hours. 

There is talways tre possibility in the group exercises 
that one individual will look relatively strong or weak in 
Some respects because of the makeup of the particular group 
en which he is placed. This and the possibility that 
SbServers May not be consistent in how they judge and report 
what they see make the process one of low reliability (con- 
BS tency) 

ee sno cextalnty that performance under testing 

E as wa transter to later performance under real 
is: particularly if results on the “correctness” 
nietas wers are Considered rather than style. 
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d. The cost of this technique varies according to the method i 
As pes” used for the interpretations of the results. 

If the empirical approach has been taken to simply validate 
some measurable results or actions or answers by comparing 
them statistically with later judgments such as ratings, 
premotions, survival, and so on, the cost of administering 
opemetreactily interpreting is relatively low. However, if 

the approach taken toward interpretation of the situational 

Ge FOD-sample tésts is onc of a commonsense review by 
management as to the adequacy and appropriateness of the 

t Ions or performance, or ıf it is an interpretation by 
psychologists of pessonality characteristics, in each of 

these circumstances it may be quite costly because of the 
hourly salary of the key individual making the interpretation, 
erat is either the manarer or the psychologist. 

e. No matter how this technique is to be judged, it should 
be employed only after a number of experimental or pilot 
Sessions havesbesen conducted and analyzed. 


Parry [52] offers insights into the reasons 
ra simtlationsese@m to be more effective than traditional tests: 


The use of games and simulations enables the learner to 
develop in three major areas. Here are the objectives that 
apply to virtually all games and simulations. 


The participant Sharpens his skills of analysis, making 
¡alerts dnd experiencing the immediate Consequences of 
them. For example, he learns to separate relevant from 
irrelevant variables, establish priorities, identify assump- 
mor e separate fact from opinion aset realisticggeals. 
ASS a le ee Weregnat chertacts aceordıne Tome 
relative importance), distinguish between might do and must 
do, and so on. Let's summarize this objective with the 


phrase Analytical Thinking. 


The participant gains insight into his own behavior as 
he interacts with others; and he develops sensitivity to 
Bie perceptions, needs, goals, management styles--in short, 
the behavior--of others. To this end, a game might be 
Meca 15 4 pro —ectave device. The participants are 
caught up in the spirit of the game and play it without 
Stomping tomaslke, “I wonder what this move will show about 
my personality? How does my performance in the game relate 
tO my plertommance on the job?" Once the game is over and 
these questions come out in discussion, the participant 
berams to realizes that the behavior he projected during the 
game is, more likely than not. an accurate sample of his 
behavior on the job. We might summarize this objective 


case IEA ana Sensitivity. 
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Par etei pant practices and refines his skills in 
DB ameseitcetively with others. Broadly speaking, we 
can divide the work of managers and supervisors into two 
Droadsareas: people handling skills (c.g., communicatıons, 
human relations, training and developing others, ctc.), and 
mot emandiing skills (e.g., work simplification, planning, 
scheduling, controlling, etc.). Both of these elements 
are present in most games. A Successful manager, of course, 
is effective on both dimensions. In contrast, the new 
manager tends to be either task-oriented or people-oriented. 
Thus, we might regard the games as aseries of exercises in 
e each Participant can successively develop and refine 
w ure Of people handling and task-handling skills. 
e e the pareseskllls. Pracsticum to summarize this 
objective. 


TWese= three objectives were brought home very nicely 
by a participant who remarked, "These games are sneaky. I 
Menus to like I™play poker or any other game... 
play to win, without giving much thought to what 'style' 
I'm using or what my alternatives are. But then, when we 
sit down to discuss the games, I begin to sce some of the 
other ways that I might have played them. In fact, I now 
find myself giving a good deal more thought to my poker 
game. I guess, in a way, that all of life is a game, and 
that how you play it is just as important as whether or not 
ou win. We Can't all be winners .. . at least not all 
Piezeıme. But we can play a good game all the time." 
Greenwood and McNamara [28] report their study 
Ge interrater reliability in situational tests." Not only 
Baa they tind acceptably high interrater reliabilities, but, 
fermen aniportantliy,; “ . . . that adequate reliability can be 
obtained from the use of nonprofessional evaluators in business- 
Piientea situational tests." 
Proscecd@nemto articles concerned with group 
Eeetedces, otreuiert, et.al. [60] make the transition with 
their description of "A Tactical Game for the Analysis of Complex 
Peerslonmvaking in Indviduals and Groups."' Their simulation 
was designed to maximize differences in information-processing 


characteristics. They ascribe the differences found primarily 


tO Variations in a personality and environment. This instrument 
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memorierily described, along with techniques for analysis of 
results, and information on how to purchase the associated 
computer program is provided. 

Herme cet Halo AAA] report om "Dimensions of 
Leadership in a Simulated Combat Situation."' This simulation 
probes different dimensions than the proceeding one. Although 
more Primarily agreponis 0 reseasneh the description of the 
siamulation-would be of value-to a military assessment program 
designer. Some idea of the flavor of a combat simulation can 
Peeobtained from this excerpt: 

After fourihoursi sleep, themotimcen was awakened at 
Peoveand told to report to MAAG Wea@dquarters. The host 
Mion had been invaded with nu@beam strikes. His task 
u Zuext eıchtslenurs Was to direct, by radio, four 
jeep-mounted teams in a reconnaissance of war damage and 
or von levels, Cöntınually receiving and recording 
information, transmitting orders to meet emergencies, and 
Pa report otr results Of the Survey to his Supcricr. 

Of all the technical and administrative tasks, this one was 
wen ed out under the most. sustained pressure and fatigue. 
Jaffee's book [90] contains concise but com- 
mete examples of both individual (in-basket), and group 
(leaderless group discussion and business game) simulations. 
Me examples are complete with scoring check-lists and forms. 
One of the sources most relevant to military 
Meplications is the report on “Development of Leadership Assess- 
Mt Simutations, by Olmstead et. al. [122]. This is a detailed 
Peet iperon Of the design of the simulations currently used 
Mene USATS Assessment Center pilot project. This work was 
done under contract by the Human Resources Research Organi- 


MAtion (HuÑmRRO) Division No. 4 which is located at Fort 


Henning. The combination of design philosophy, problems 
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encountered and their solution, rating, analysis of results 
and assessor training, make this reference especially valuable 
as a guide for a program designer. 

The in-basket exercise is the most widely used 
migavidual simulation and the literature is replete with 
w icles on this topic. The stature of the exercise is illus- 
peated by this quotation from Byham and Pentecost [12]: ''How- 
mver, one exercise, the an-basket, seems to have emerged as 
Mes most important exercise of most assessment centers." This 
ferew represents a consensus expressed by other writers. 

Frederiksen, et: al: [122] provide a description 
of how the exercise came into being and its early development. 
Examples of in-basket materials, scoring and validation studies 
mee reported. 

hes landnanksarticle gn the in-basket fields 
Bopez' “Evaluating executive decision making," [119]. This 
source covers the entire spectrum from design considerations 
to application and analysis. It also covers the previously 
mentioned study along with others done by: The Bell System, 
the Bureau of Business, Harvard Business School, Port Authority 
me New York, Sears Roebuck, and General Electric. In scope and 
detail, it is unmatched. 

Meyer [48] details the design and validation 
of the General Electric in-basket. Although the test itself 
7S not included, the article would be an excellent resource 


for the program designer. 
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Jeticesotters two books contalgang in-basket 
information [90], [89]. The latter is devoted exclusively to 
the in-basket. It is a workbook containing two complete in- 
basket exercises with all forms, instructions, etc. to enable 
Bam tratıeseto Individuals? This 186 not to amply Ghat it 
1s intended to provide the program designer with an off-the- 
Shelf instrument, for the point is made repeatedly that to 
be effective the in-basket must be sd to a Sspcert te 
rocam.: Rather, the book was intended as a training device 
for students. 

f. Reliability 
The summary of reliability studies done by Howard 
[34] negates the requirement for considering them individually. 
phe writes: 


In many assessment center exercises and in the final 
evgmwations Gach participame issevaliated by mome thenecne 
a, Or. Accordingly, interrater reliability Becomes a 
Meter Fot some importance, in addition to tne reliability 
of individual measures. A summary of reliabiiity data Is 
Seowm in fable 1. 


Bt should be noted that the AT! studies of the in- 
basket, projective tests and interviews probably had 
infiated reliability estimates, since the interrater relia- 
bility was determined for the written report of a procedure 
not the procedure itself. Two raters in high agreement on 
what a report says is a far less potent finding than two 
raters in high agreement on how a candidate performs in a 
Tenn tional exercise. the reliability coefficients do 
Bl cate, however, that the reports presented clear evalu- 
ations from which consistent ratings could be made. 


In summary, based on the data available, interrater 
reliabilities for assessment evaluations and for several 
assessment components seem sufficiently high to support 
their further use. There appear to be no advantages of 
ratings vs. rankings or psychologists vs. managers in terms 
om reliability. 
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0. Validity 
Byham's article [11] provides a basic primer on 
validity Mi mens arenas At the other end of the spectrum, 
mrt [61] provides a wide-ranging, probing treatment-of the 
MI With respect pemthescriteraon problemehe writes: 


All assessment programs involve studies of the link 
wetween two Or more pieces ot behavior, whether the primary 
purpese be selection, validation research on tests; or 
Personality research. Some of this behavior is known as 
assessment behavior and some as criterion behavior. These 
concepts are analagous to the independent and dependent 
variables in experimental psychology, and it 1S an arbi- 
brary deci silenebythiermexperimentver whichwone wedesignated 
MEE. Most ot tie reports of assessment have devoted some 
Sete to the Criterion problem 


A special problem that arises in personality assessment 
u he trequent unmelmability of thescriterionswhaech so 
D cen nepresent subjectivemudaments tluat many Irom one 
Siitervon rater to@Manetiter. Whis unreliability mpostes a 
Poetous linita@eiom on the potemtial*walidaity of persenaktity 
assessments, and it makes it difficult to evaluate some of 
miner Low Validity coefficients reported 


The assessment strategy should be aimed at the criterion, 
ice the latter has beens established . . . 


A complication that arises in criterřa analysis, 
we thatan "assessor can only#prediet to dices of the 
criteria, not to the actual criteria, themselves 


Mlculiles tne —a5ceasors May be able to convince those 
Who control the criterion ratings that the indices which 
e latter are using are not consistent with their funda- 
Memcal criterion, but eventwally tle assessors and the 
criterion raters must agree on some criterion index in 
mecomaaqmee with thepellicy@or the Grcanization. . 


A OE chines Peeing ccud the best assessors for 
Predicting existing Criteria are those who are partially 
contaminated with the same experience, standards, and 
MT Toc dee thes Criterion raters and can thus rely on a 
o Dallas ba tegar to make their predictions. 

Mt also addresses thewfact that criteria are dynamic rather 


fhan static: 
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A further condition that 1s often ignored in assessment 
Pomtilat Of Ciiluxion of time; the predictions are usually 
made on the assumption that the status of the candidate 
on the relevant variables will remain constant over time 


INS mes complex situations, in which the criterion 
performance is highly dependent on the conditions, the 
inability of the assessors to predict the specific condi- 
plans that will operate for any particular candidate may 
render the assessments completely invalid. 


This latter point is restated and amplified by 
Ghiselli and Haire [24] who write: 


When we validate selection tests, we typically relate 
scores to some measure of job performance obtained during 
a brief initial period of employment. This initial period 
may be as short as the first two or three months of employ- 
ment, and it seldom is as long as the first two or three 
years. Presumably the men are hired for a much longer 
period the rationale of our validation procedures involves 
matching a predictor variable with a criterion. But, does 
ioe cr TeCT Ion) itself predict the one we are interested 
Denon mance over a relatively long period? To be sure, 
eee Cores Tor the initial period indicate the con- 
Prrputtom workers have made during that time, but this is 
not exactly what we want to measure. At least implicitly, 
Perry eE iterion date are taken to stand for ultimate per- 
acento totale peritormance on the job. The period 
selected for a criterion is usually determined by practical 
considerations such as the availability of data. The 
question to which this paper addresses itself 15: Do tests 
enanada ted against such a brief initial criterion maintain 
their validity over a longer period of measured performance? 
There are indications that they do not . . . Worbois [72]. 


Ip rTactice of using performance data obtained during 
mmni tial period and letting it stand for ultimate or total 
pertormanee Completely ignores the dynamic character of the 
Criterion and important changes that are taking place in 
bhe kworker's performance. Over a considerable length of 
MNR ICE OD there 1S not only a change in average per- 
formance, but also significant changes in the order of 
individuals in their performance. The change in average 
performance may be much more than we are used to thinking: 
we are used to laboratory tasks which can be mastered in 
an hour or so. On the job, performance may improve for 
many momths or even years. . . 


In one case, the authors have followed the progress of 
a group of investment salesmen for ten years. During this 
Beetended period there was 650 percent improvement in aver- 
age productivity and still no evidence of leveling off. 
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Indeed a straight line would fit the whole data well. If 
there are significant changes in order within such a period 
of improvement, early validation will be scriously mis- 
leading. A good many studies have indicated such a change 
in order. Contiguous stages of learning typically correlate 
much higher than periods separated by some time . . . Fac- 
torial analyses of learning data clearly indicate a change 
in weighting of factors for different periods of the 
learning process . . . All of these things suggest that 
early criterion data do not measure all of the important 
variables in performance. 


With the average performance and rank order of individuals 
both changing, one is tempted to turn to the rate of improve- 
ment as a criterion. If ultimate or total performance is 
Our goal, rate may be a useful substitute. To wait for 
final performance is impractical; even maximal performance 
may be so far off that it delays the validation of new 
instruments impossibly. There is some evidence that final 
mevel 1S predicted by rate of improvement, so, even though 
it indicates little about an individual's level of perfor- 
mance during the initial period, it may be more indicative 
me final values than the early criterion itself. In view 
Or the difficulties encountered with a partial criterion 
mich does not predict the ultimate criterion, this study 
ae ita particular case, the validation of tests 
for the prediction of differential rates of improvement. 


From their study of 56 taxicab drivers, Ghiselli and Haire 
arrived at the following conclusions: 


i lc soe. tod tad it tile sdyialle nature of the 
Criterion of job performance presents real difficulties 
in using initial measures. If the desired criterion is 
Mmitdmate or total performance, there is some question 
whether an initial criterion measure will itself be a 
ood predictor. If not, validation against this early 
criterion is no validation at all. 


2. Data are presented showing, in a particular case, 
mob pertormance over a period of time. During this time 
weni icant changes occurred in the average performance, 
the variance, and the rank order of workers. 


3. Validating a battery of tests against the criterion 
re obtained, it is possible to predict the first three 
Meeks ot performance, or the last three weeks, but it is 
Pete veeetDle tO predict both with the same battery. The 
tests show various patterns of validity: low validity in 
meetial periods, rising, later; high validity in initial 
Mods, falling later; and cyclical variations in valid- 
MN measured against successive stages in performance. 
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MI ie seradata, eet 1S possible to predict rate of 
improvement with some success, but, over the brief period 
investigated, rate 1s not strikingly associated with total 
production. 


Bray and Moses' [76] review of the literature 
produced these comments regarding the criterion problem: 


Such heightened concerns about test validation led 
A tO tie. Criterion problem,’ a recurring problem 
through the years because few researchers directly face 
üp to this issue. A little progress was noted. For example, 
MN SOL Y alos, one of the most easily obtained cri- 
lemme measures, asain were COnvincingly discredited, which 
no doubt will discourage few from using them in the future. 
An implicit point also under fire was whether criteria used 
in various test validation studies were actually pertinent 
to the test validated. 


Potion wIMpOt rant slsslc Qi the past three years is related 
Ceci Ooms ection n a different way. This is the 
question of how long after selection a criterion measure 
should be taken. This concern, as well as a somewhat 
newer interest in developmental changes in employees has led 


to even more urgent pleas for longitudinal studies than was 
ieee rey OUSLY 


One of the more impressive analyses was presented by 
Ronan and Prien . ... [123] . . . who appear to be moving 
I seroser to a theory of eriteria . . . Their review of 
the literature demonstrates that it is totally unrealistic 
to assume that job performance is unidimensional. A singie 
criterion, even a composite one, is often very misleading. 

many investigators have not been attentive to the 
multidimensional Nature of job performance. Instead they 
have too often used existing, readily available organiza- 
E1TOnalmemnaicesma! lececa fo be indicative of performance 
rather than devising measures focused on appropriate 
Deliawmror. . 


What we are Saying is that some very serious questions 
must be raised about the efficacy of the continued and 
erea use of Supervisory ratings as the criterion 
measure. Perhaps this is why we have a criterion "problem." 

HoE eaa ele: "Ihe validity of athe in- 
basket as a measure of managerial performance" provides a 


description of test design and validation. lle also performed 


a Cross validation in his study. 
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Guion and Gottier [29] provide a review of "Validity 
Of personality measures in personnel selection" for the twelve 
Bear period 1952 - 1963. They presenta tabular summary of 
95 validation studies done on fifteen different personality 
and interest inventories. Similar summaries are included on 
Erojeetive measures, special inventories, and personal history 
data. Unfortunately, their conclusions are not very encouraging, 
as they note: 


Dnerealessno ,. « .seviudence in the survey that companies 
. . Nave. . . recognized the need to validate the instru- 
ments chosen for their own specific situations . . . 


Even without these flaws nearly every study may be 
criticized as following the traditional paradigm of testing 
mora relationship between the test and a criterion. Cer- 
tainly more imaginative designs might well reflect the 
undoubtedly complex relationship between motivational 
Variables, situational variabies, ability variables, and 
behavior. 


miecy also echo an often-mentioned bit of advice for the program 
designer: 


Whatever the reason, the point still remains: a home- 
made personality or interest measure, carefully and com- 
pletely developed for a specific situation, is a better 
bet for prediction than is a standard personality measure 
with a standard system of scoring. 


This review of references concerned with validity 
1s concluded by Howard [34]. She provides tabular summaries 
SeeeseVerdde Valadityestudazes „was fobkow ss 


Meet muse De regarded as “The Study" in assessment 
center validity is ATGT's Management Progress Study 
phiteh@was predictive and “wncontaminated;'' 1.e., results 
were retained for research purposes only and not released 
Pomanagement to influence promotion decisions. The 
researchers administered the assessment procedure to 422 
male employees of six Bell Telcphone companies beginning 
in 1956, stored the results, and waited eight years before 
pursuing information on the assessee's progress in the 
company. While many may view eight years of waiting as 
m almost unbelievable display of forebearance, the authors 
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admit that by their own standards they were impatient-- 
they had intended to “wait ten years, Comparisons made in 
1965 of management level achieved by men assessed six to 
eleht years previously are shown Tn (Figure 9). Validity 
for the assessment predictions was amply demonstrated. 


Eroume 9., 


Relationship Between AT&T? Assessment Staff Prediction 
and Management Level Achieved 





Predi gions make Se CUS Stn U1 Vie 1 96S 
middle management % lst level 6 2nd level 6 middle 
within 10 years management management management 
| a 
Yes Coleco 
(N = 62) 2 50 48 
mo or ? 
(N = 63) 11 78 eet 
nes Non-College® 
(N = 41) í 61 2 
mo or ? 
(N = 103) | 60 55 5 


Note: Adapted from Bray and Grant [10]. Source: Howard [34] 


*x° significant at p <.001 


Point biserial correlations were .44 -for the college 
Poupeand ./1 for the non college group. Of the total number 
of men who reached middle management, 78 percent were correctly 
Mmewitlitedaby the assec sment stali. In contrast, among those 
in both groups who had not progressed further than first 
mevel Management, the assessors predicted that 95 percent 
would not reach middle management within ten years. Note 
m ese prediction e ilil had two years to run; later 
communications from the company indicate that even greater 
accuracy was achieved. 


Correlations hetween assessment ratings of general effec- 
tiveness and salary increments were also given for four 
samples of individuals who had "at least six years of tenure 
in management since being assessed; (a) Company A, 54 college 


men, r = .41; (b) Company C, 27 college men, r = .51; (c) 
Pompany B, 83 college men, r = .45; (d) Company C, 39 non- 
Sollepe men, vr = .52. 41] correlations were significant 


MU Combined With the data in Table 2, the useful- 
mesemoe AlGl's overall assessment ratings for predicting 
management success seems well established. 
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Another AT&T study with newly hired candidates for sales 
positions also used an uncontaminated, predictive validity 
womaanmone (5). The primary criterion of performance was a 
six-month field review by an cxpcrienced team from AT&T 
headquarters which regularly makes such inspections. Where 


the assessment judgment was ''more than acceptable," 100 
Noche “Sabesien met the review Standards. Comparable 
Success figures for those judged "acceptable," "less than 


acceptable," and "unacceptable" were 60 percent, 44 percent, 
Mer cent, respectively, producing a chi-square value 
APS . 008). Aga the predictive validity of 
AT&T's overall assessment was evident, this time for the 

job of salesman and over a short time interval. 


Propirens* of "eri eT Pon consaninetecneliave confounded 
Scala studies other than eiose described 
above, Since assessment ratings were used in promotional 
decisions. Where assessment ratings were used primarily 
emale the first promotion, the effect on later promotions 
was not felt to be large, however. A summary of these 
Sen taminated criterion studies isshown in Figure 10. 


Paviiethietsiiates done, Co date overall ratings of 
potential or performance from assessment center procedures 
penerally have shown impressive predictive validity, 
especially for managerial jobs. Unfortunately, use of 
the ratings for decision making about assessees' careers 
Berewiat rTestrains an overwhciming acceptance om tlic 
findings. Nevertheless, "clean" predictive validity has 
been demonstrated, but only in two studies with both at 
the same company. 


h. Peer Ratings 
Kraut [39] reports an IBM study on peer ratings 
conducted on samples of managers and cxecutives. In summary 
he writes: 
eS Du ness Noria. as elsewhere, people regularly 
form impressions of their associates. The use of peer 
ratings among managers may merely formalize a process in 
which managers already engage informally. This study ind1- 
Gates that the use of managers' peer ratings is likely to 
Ns ela Der od men who move into the executive ranks. 
A source which deals with a more relevant appli- 
Sreloneus stat by Hobland§r)[55]|]. His paper reports a study 


Roo Ue, S. Naval Officer Candidates circa 1955. He 


MEDOTtS: 
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Peeeweceice presented Uniformly supports the con- 
clusion that peer nominations used early in training can 
Perea atstimctive contribution to the prediction of a 
Me alae Criterion of performance after training. A 
fea eet implication Of this is to encourage the judicious 
Mee@wotesich Garly evaluations as a supplement to other 
predictive measures. 


Ane nai tary application of the use of peer 
nominations (PNs) is reported by Amir, et. ai. [2]. Their 
Study was done with soldiers of the Israel Defense Forces in 
1964 - 1965. <A total of 3897 subjects were involved. Their 
Ma SiO the results were summarized in the following 
paragraphs: 


What explanation can be found for the high PN validity 
Seoctticients reported here in contrast to those reported 
1n earlier studies? Moreover, what explains the higher 
Ate nt reported Lor promotion to officer rank from 
It ses than those predicting promotion to NCO courses 
Bone ee eraıning? .. . . In respect to the former question, 
Eur sro loyıne reasons appear relevant: The present inves- 
tigators observed a high level cf task orientation displayed 
by the soldiers who responded to the PN questions. Their 
attitude may be a function of the knowledge shared by most, 
PPro eal, Of che troops that their present or future 
commanders may be called upon at any moment to demonstrate 
teeta eleadership abilities in actual combat. Therefore, 
een cend to choose those of their companions who really 
impress them as possessing these qualities. Incidentially, 
this high level of task orientation may be inferred 
Mae trom the lack of contamination of the PN scores 
ends ai choices. The validity coefficient of PN 
eee coma ter Tr tenaship Scores are partialled out was .55 
meter ca tlm oo fOr "basic training prior to the partially 
Ine tt1cndsnipn Score and ./8 as compared to .79 
for NCO courses. | 


ISS cOn possible explanation lor the relatively high 
Meet aens reported here lies in the Pact that soldiers 
accua iited with the army's criteria for promotion 
ce rank: This information appears to be widely 
disseminated among the troops as a result of the close and 
Wie ormmilecontact Maintained between soldiers of all ranks. 
aime tationships facilitate communication between 
e O e, MEnese reasons appear to be more potent for 
POTS iiene) courses than for those in basic training, 
which is perhaps why predictions regarding promotion from 
IES Tank are Of preater validity than those from 
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Pewee co NCO rank. Troops in training for NCO 
rank appear to identify more closely with the values and 
requirements of the army than do the recruits, and are 
already an integral part of the military leadership. Con- 
Peaquemely, they have more understanding of the qualities 
and demands of leadership than do newly enlisted troops, 

so that the PN choices of the NCO trainees are likely to 
have greater predictive power than those of troops in basic 
Eraining. 


Mhen inal selection c hosen relates directly to 
the assessment center use of peer ratings. Finkle and Jones 
[85] provide a discussion on the use of such ratings from a 
design standpoint and cover some of the advantages and dis- 
advantages of including peer ratings in an assessment program: 


Den tunatıcnalacr zercisesy we expect that peer ratings, 
at they are to be used, have to be constructed for any 
Ases sment program 11 accordance with ties fommat and objec- 
ESO che program... Ihe peer, or Socwemetric, ratings 
can provide the assessment committee with information as 
to how the assessees themselves judged or viewed one another. 
Puch @etOrmation can be solicited at the end of the program, 
Ames GUring the program, or both. Questionnaires can 
be designed to cover opinions about particular parts of the 
Peomran, Such aS a group exercise just completed, or about 
Overall skills or characteristics witnessed throughout the 
program. The manner of questioning can be as varied as 
oily formol rating iMWeluding asking assessees to rank one 
another (including or not including oneself) on overall 
E outon Co an exercise or on specific characteristics 
asking them to write out opinions or explanations about 
meir own performance or that of others, asking them to 
Tate all other assessees on scales similar (or not) in 
content to the variables the assessment committee will later 
Amploy, or asking them to nominate one or More persons 
from the total assessment group as being the best in certain 
areas. 


Obviously, the perspective of the assessees will not be 
like that of manager observers in several respects. Most 
importantly, they will simply not have been paying as much 
attention to one another as to themselves. This means they 
Really "have a limited basis for rating one another. Then, 
too, their judgment as to the significance of certain actions 
and abilities may be less realistic than that of experienced 
managers. Finally, peer ratings are always somewhat suspect 
to popularity contests (despite some clear research evidence 
to the contrary)--though absence of exposure to one another 
prior to the assessment program should forestall this pos- 
Sapility considerably. 
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i. Assessors 
Me cthiemoUciediscussion of the selection and training 
of assessors is provided by Byham [11] who writes: 
Choosing the assessors 


pue. assess®rs arezlıne Mamacers working two or 
three levels above the man being assessed. .. . These are 
the individuals who are responsible for promotion and who 
know most thoroughly the job requirements of the positions 
one level above the candidate's. 


Lic aitebaekeround of the assessor, of course, depends 
Mime purpose OL the specitlic assessment center. Where 
broader management aptitudes are being assessed, it 1s 
Benmenmeror the assessors to be drawn from a number of areas 
im a company. This not only brings in a number of viewpoints, 
mut exposes the candidate to representatives of a number 
of areas where he may find promotional opportunity. Having 
fmepresentewmaves of different areas also increases the 
Becepwanceworwthe fimdings throughout the company. 


Assessors from management, like the candidates them- 
mines are usually nominated by their superiors (although 
in a few companies the center administrator makes an 
Fort to recruit them). Naturally, the practice has its 
dangers. After a center has passed from the experimental 
pes theweperational phase, ‘‘purity"™' controls may be relaxed 
somewhat, and senior management may be tempted to send 
pecoperative'" managers to centers to act as assessors. 
mms temptation WS particularly strong where the assessors 
Serve for extended terms. 


Center administrators have chosen to react to this 
problem in various ways. Some companies rely on their 
assessor training programs to screen out assessors who 
are ulaeceptable in the role, for one reason or another. 
Tice rectwlonale@here 1s that it is easy to spot an unquali- 
fied assessor during training and ease him out without 
bloodshed. As a fine point of strategy, for example, many 
Beiter aamanistravems’ suggest@that Iit is wise to establish 
wool ot assessors, rather than train asssessors for 
Emecitic assignments. With the pooling arrangement, it 
mancaSy for the administrator to bypass unqualified 
assessors. 


a Ont oF controversy among operators of assess- 
ment centers is the desirability of using professional 
peycholepasts rather than specially trained managers as 
assessors. Most arguments for using psychologists are 
based on their skills in observation; they are trained 
to recognize behavior not obvious to the untrained eye. 
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While this argument is plausible, it has yet to be demon- 
strated in an operational center. Three studies have found 
no differences. 


However, the superiority of psychologists over completely 
untrained managers is well established. Because of this 
Superiority, companies often use psychologists as assessors 
in experimental or pilot programs, where training management 
Besessors would be difficult. Psychologists are also used 
extensively for assessing higher levels of company manage- 
mateo e miel levels, it 1s difficult to pet and train 
managers who do not know the candidates personally, and 
the objective, independent psychologist is seen as the 
fairest evaluator. i 


By and large, companies now prefer to establish a pool 
of trained manager-assessors, each of whom serves more than 
once. Individual assessors are usually drawn from the pool 
to serve once or twice a year--a few companies ask assessors 
erre soaly once n Alal s practice is exceptional --it 
assigns assessors for six-month terms and center admini- 
Strators for one year. 


There are advantages and disadvantages to brief assign- 
ments. On the one hand, brief assignments usually mean 
that better men can be recruited, their enthusiasm and effort 
Delimbe Oreacer, morc Managers will benefit from the training 
involved in becoming an assessor, and more managers will be 
well prepared, after their tour of duty is over, to make 
¡Másczous use of assessment reports. On the other hand, 
more managers must be trained and kept off their jobs, and 
those who serve briefly will not have as comprehensive an 
experience as assessors as they would if they had served 
a longer period. 


Where the appointment is for an extended period of six 
months or so, of course, more rigorous and lengthy assessor 
training is feasable--AT&T trains managers for a month-- 
and longer experience in the role is very valuable to an 
assessor. One substantial disadvantage of the long assign- 
ment is that assessment becomes a routine matter, which it 
never should. Reports from fatigued assessors read like 
computer output, and it is hard to think of them as anything 
more. Currently, only AT&T appoints assessors for prolonged 
periods. 


Training the assessors 


ime tne Companies now Operating assessment centers, there 
mea motable difference in the emphasis placed on training 
ISS SOSA Same Companies pive new assessors as little as 
ARNO UY ol training, which really amounts to just an 
orientation to the whole procedure, while most others spend 
Mires or four days. 
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One can argue that the task of an assessor is similar 
to the requirements of most managers! jobs--a manager must 
interview individuals, observe groups, and evaluate presen- 
tations. Assessing requires skill in these Same areas, 
and hence many feel that there is little justification 
hor tuther training. 


Thesprincempalr rebuttal to these arguments is this: 
because a man has been doing something, he has not neces- 
sarily been doing it well. Companies reported marked 
Piprevemcirusam= tne reliability of supervisory ratings 
after the supervisors have been trained to work as assessors. 
Nonprofessionals need to be shown what to look for in 
observing group discussions and individual, presentations, 
COMO May Lo us onspunely súurtace chamacteristics. While 
a e e tUd esmar Jackınassıt Ls obvious from 
Comparing the reports presented by experienced and inexperi- 
enced assessors that training makes a very big difference 
in the quality of performance. 


Themo s tC common method of training is by understudy. 
m enua enan, an assessor in training sits through 
Seenen eras ee amen Cycle 2s a nonvoting member. Another 
method of assessor training, particularly when assessment 
centers are being introduced, is to have the assessors go 
through the assessment experience first as candidates. 
Everything 1S the same except that there are no assessors 
present JIteastyptecalotraingne situation, the assessors 
po through an activity such as group discussion and then 
critique the discussion and identify possible areas of 
observation afforded by the situation. Several companies 
ideoi ne activities to give assessors practice in making 
observations. 


Byham and Pentecost [12] repeat much of the 
Foregoing information, and add the following points: 


Some companies believe mtNatetiliseexperience 18° so bene- 
ficial that they have gone to a one-to-one assessor-candidate 
Zatıesansorder,to expasesmeore people to it. 


Almost all of an assessor's training and experience can 
be applied to his job and should improve his ability to inter- 
whew and appraise his subordinates. 


Greenwood, et. al. [28] make the following comments 
concerning assessor selection and training: 
Ne nie cons to the evaluators, whether profes- 
Sonar imnonprotessioneal, should be explicit as to the 


estes t, oe Of behavior to be evaluated, examples provided 
as to kinds of behavior that may be expected to be elicited 
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by the specific situational exercises, and standardized 
rating forms provided for their use. However, the results 
Bemagsto Indicate that the selection of evaluators for this 
type of assessment need not be based on whether they have 
completed an extensive training program or have professional 
experience in personnel assessment. 


Taft [61] makes a strong point in support of line 


MENMTagers as aSsessors. 


All other things being equal, the best assessors for 
Preareting Existing Criteria are those who are partially 
contaminated with the same experience, standards, and 
outlook as the criterion raters and can thus rely on a 
global strategy to make their predictions. 


He makes two more pertinent observations in his summary, e.g.: 


The assessors should be selected for proven ability to 
make accurate judgments in the assessment situation, i.e., 
hey Should be valatdawed. 


The assessors should be familiar with the criterion 
Situation, and should take this situation into account 
mirem they make the predictions. 


Concerning the alleged “side benefits" claimed 
for assessor experience, Howard [34] makes the following 


mrrtical observation: 


Training Assessors--Benefits of assessor training have 
meen Claimed not only in the form of a partial solution of 
the criterion problem but through (a) improvements in 
interviewing skills (b) broadening of observation skills 
(c) increased appreciation of group dynamics and leader- 
ship styles (d) new insights into behavior (e) strengthening 
of management skills through working with simulations, and 
(£) broadening one's repertoire of responses to problems. 
No well-designed training studies have validated these 
promises, however; as has been pointed out previously, 
firms do considerably more management research on selection 
than on training and development 


In summary, Bender [3] compiled the following 


information from his survey of assessment centers: 
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Figure 11. 


Selection and Training of Assessors 


Number of organizational levels of assessors above assessees:** 
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Response frequency 0 18 7 4 
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mM One organization indicated that only psychologists and staff 
personnel were used as assessors. 


Source: Bender [3]. 


foeeeealuatson of Candidates 
Byham and Pentecost [12] introduce the topic with 
Met overview Of tlre evaluation process: 


The professionalism of an assessment center is probably 
best measured by forms provided to assessors to record and 


report their observations. These can range from a detailed 
Meme iption of all pertinent behavior to a series of 1-t#e-5 
ratings on key variables. Well-developed forms can guide 


the assessors in making observations and can aid them in 
Mel uring reports. 


There are several ways of reaching assessors' consensus 
onthe candidates, among them a method based on predetermined 
factors. Discussion centers on one candidate at a time. The 
assessor reports to the group on the exercise he has observed; 
all assessors then rate the candidate 1 to 5 on the list of 


factors. This procedure is repeated for each exercise 
reported and the assessors then rate the candidate's overall 
performance on each factor. Next, each assesssor announces 


his rating on each factor and differences are discussed 
nei] consensus is reached. 
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iiemadvancage ot the factor-rating approach is that it 
forces the assessors to consider as important the whole 
range of behavior identified. Otherwise the assessment 
decision might fail to consider impertant areas because 
Sie overriding posa tive or negative performance in others. 
MieEo, the assessment decistens tend to be more organized. 


Another approach is for assessors to give reports on 
a candidate, one after another while all assessors take 
notes. A general discussion follows. This procedure 
facilitates the integration and interrelation of obser- 
vations, but may result in so much data that 1t is dif- 
eult for theassessors to consider all of it at one time. 


Stiil another variation is to have one assessor write 
a composite report on a candidate from individual written 
reports prepared by assessors after each exercise. The 
composite reports are then discussed by the entire group 
of assessors until consensus is reached. 


A more thorough treatment is provided by Finkle 
and Jones [85] as they describe the procedure used in the 
ohio program: 


Accepting the philosophy that we learn best by doing, 
we begin the evaluation of our first assessee after having 
Spent approximately an hour and a half on orientation. 
foment the possible exception of the first case, the assessees 
mee cons ldered In alpnabetıcal order. An attempt 15 made 
by the staff to select as the first case one that will 
probably prove to be rather straightforward and consistent. 
he do not feel this presort is critical and a quick scan 
may not, obviously, accomplish the objective. However, 
the management representatives gain further insight into 
the variables by actually putting them into practicc, and 
we feel that it is desirable to avoid the confusion that 
may arise when discussing an unusually controversial 
amy dual. Of course learning can and does occur when 
there is a good deal of aifference of opinion which arises 
because of the complexity of the assessee's personality. 
However, understanding how to use the variables is dif- 
ficult enough in the first case without adding further 
complications if they can be successfully avoided. 


The same procedural format is used for all evaluations. 


1. Pictures of the assessee are circulated among the 
Staff members to refresh memories and to be sure that those 
who have seen the assesseeS in operation associate the name 
Pao ace correctly. A rating booklet and a completed 
assessment summary sheet containing the relevant test results 
ento each staff mémber together with a copy of the 
biographical questionnaire the assessee filled in prior to 
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the program and a set of note paper for making notes on 
reports presented by cther members of the assessment 
Committee. 


2. The manager who conducted the personal interview 
of the assessee being revicwed presents his report of 
tor rmation which complements the blographicel questionnaire. 
He also relates the general manner in which the assessee 
handled himself during the interview. 


3. The psychologist who interpreted the projective test 
materials reads his report. This report focuses upon the 
way the assessee structures and organizes his work, his 
attitudes toward people, his apparent motivation, and his 
emouronal stability. 


Mote: Programs employing projective tests are in the 
Minority.) 


wo lew cle Trepert Of the assessee's performance in 
PiciMmoeestiticmsltldttomnal cxer1cises 1S presented. Each 
ess comas been observed by a different management repre- 
sentative at each exercise, so that a report on him is made 
by each of the management staff. Responses of the assessee 
to a questionnaire completed after each exercise are read 
to the committee by the program administrator. The tables 
Samaria thie peer rankings of effectiveness which are 
completed following each oí the exercises are handed cut 
with a reminder to keep them through the review of all 
As essces as well as the first. 


Dee ciscussion Is held of the appraisal by all con- 
mittee members of five-minute talks that were given by 
each assessee. 


ne report forms showing the results of a final 
Peer questionanire completed by the assessees as their 
Wst task during the three-day assessment period are dis- 
tributed--again with a reminder to keep them through all 
assessee evaluations. 


Durane the presentation of information, the committee 
members are encouraged to ask questions as each report Is 
meesencca, not only to Clarify the specific information 
being presented but also to add further knowledge about 
Ae Procedure and the nature of its contribution to the 
me O@iam. this need for 2 full understanding of the pro- 
E o esns a ticularly trues of the projective test 
Ios kely rhis is the management representatives 
E a a h the concept of projective testing. It 
a an that they know something of the theory under- 
D a eand the limitations of this type of testing. 
ais reports are fully discussed and the presenters 
questioned, the focus of this discussion should be 


t 
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Nation rather than evaluation as such. Either the 
assessee or the particular procedure in use should be better 
known as a result of the questioning. The professional 

ass the responsıbilatyito see that the committee avoads 
premature efforts to "type" or draw inferences rather than 
give Or digest a descriptive report of findings. 


Ice personal history has been reviewed, the inter- 
view, projectives, Situational exercises, and questionnaire 
information presented, and the five-minute talk discussed, 
the assessment committee has available to it the evidence 
upon which it will base its recommendations. The task now 
is to draw upon this information to judge the personal 
Bnaracteristics in the rating booklet. 

The objective is to build a consensus of opinion con- 
cerning how the assessee behaves as a person as a starting 
point for the committee and other users of the information 
to consider the implications his talents and style have for 
ARIS iUture contribution to the company. The committee 
members are instructed to rate the scales on the basis of 
their judgment after considering all the information. 

They proceed one scale at a time. The individual members 
make their ratings independently and announce them upon 
request of the committee chairman after all have had full 
Bepertumtty to arrive at their scores. Each committec 
member simply calls out, upon request, the scale point 

Kor example, 2, 2-1/2, 3} which he checked in his own 
rating booklet. The chairman tallies the ratings and 

calls for a discussion of any discrepancies greater than 
one point in order to assure that all committee members 

are Weippe all relevant information, excluding irrelevant 
information, and attaching the intended meaning to the 
variable being rated and the bench-marks (scale descriptions) 
on the scale. If the review of sources and scale meaning 
persuades one or more of the committee to change his mark, 
the corrected mark is made in red and the committee chair- 
man 1S informed. If no one wishes to change his mark, the 
discussion is closed and the next variable is considered. 
This variance in committee opinion is then reflected in 

the narrative report. 


Because the first case 15 a learning exercise as well as 
a practical handling of results, it is well to avoid the 
temptation to rush. Time is taken to call attention to the 
Mearious sources of information throughout the protocol 
that support the ratings the assessment committee has arrived 
me. much of the discussion is focused on the scale derfini- 
tions so that the concepts will become better crystallized 
for the management representatives. For, as understanding 
meereases, the frequency of differences in ratings which 
mecur pramiarily because of differing interpretations of 
the scales' meanings or the behavior indicated by the 
Seale points is reduced. 
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We have found that 1-1/2 to 2 hours can be profitably 
m initial Case. The time required for subsequent 
cases falls within the range of 1 to 1-1/2 hours. We see 
no evidence that a reduced amount of time spent on a particu- 
lar assessee has reduced the quality of assessment. Our 
observations, admittedly subjective, lead us to believe that 
the feeling of responsibility of the assessment committee 
members causes them to adjust their pace appropriately to 
the complexity and needs of the case they are reviewing and 
in accordance with the confidence they have in their personal 
understanding and mastery of the system. 


One of the procedural problems we have been alert to has 
been the possibility that the interpretation and integration 
of the cases may vary as a function of the sequence in which 
they are introduced in the integration session. Yet we see 
Mo evidence that having an assessee discussed early or late 
Mmimtniew session has a Significant bearing on the quality or 
Pent) sO tie ratings or recommendations. Repetition of 
Mas t Case as a thirteenth case has indicated that 
wetmemrinrst did not affect the outcome. Admittedly, there 
Mae been times when one or more members of the professional 
staff has had reservations about the quality of the committee's 
judgments or recommendations with regard to a particular 
assessee. However, we do not see such instances as related 
sequence of consideration, to length of time devoted to 
Miemease. OF to añy other procedusal factor that might be 
aaa ados ted, improved, or controlled. 


Following the completion of ratings on the various scales, 
teme rap transparency 15 used to project on a screen a 
summary of the talliesmade by the program administrator of 
ene Committee judgments on each scale. Also on this slide 
Ee Now or the first time--the "pipe organ" diagrams 
showing the results for that assessee on the mental ability 
tests. When these scores are noted, either one member of 
the professional staff or the program administrator reviews 
fOr the committee the ratings given on each of the scales. 
Mewmewiieg the rating scales, of course, Serves to consoli- 
date the group's thinking about the assessee so that all 
members view him in a reasonably comparable way before con- 
dering the questions of placement, development, and 
potential. We have adopted the practice of withholding 
information about the assessee's overall mental ability 
maca directly from tests until this time to avoid 
the possible biasing effect of this information on the 
iecements Of Other characteristics and skills. This factor 
aS then introduced at the time the judgments are reviewed 
mice ts considered in the general comments, the placement, 
Severvisory, development and potential recommendations. 
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Resolving Differences 


The task of working toward a resolution of those dif- 
feremees Of opinion that inevitably arise in the rating of 
the behavioral variables warrants discussion before we 
tiene tO thie process for developing the final recommendations 
Semecerning Utilization end development. In actual prac- 
Be, 15 the Committee Chairman whe exercises the key 
role in determining whether an attempt will be made to 
meoceve dlttemences Successinuily, at feast to the satis- 
faction of the disagreeing committee members, or whether 
issues will be merely politely debated. However, if the 
debate or discussion moves into a somewhat complex con- 
hioeratloln OLspersonalszty theory, hunan behavior, or social 
Sc Tpactions” Che Domenole@cistS present wall naturally be 
looked to as the experts who should resolve the matter. 


Lese tzomegzeawikier tie pressure upon the professional 
o embers to speak competently as experts at times and 
[Me Meletse JUCTCTOUSIY their responsibilities to aid the 
Deae eene representatives in the anterpretation of appar- 
ently contradictory test findings. If the professionals 
Acto Caretill-, Newevel.eblicy Callmaspercelve their 
Ap ropriatel felt pressure to pertorm in a professionally 
Sound manner as a mandate to operate infallibly. Yet we 
have discovered that a proiessional staff person can be 
Peo Meer eCh iets Trege@iiemand-wnduiy SelecLive in 
his handling of the information available about the 
ae eet SOL COlmse “expected that this will no 
ore case. Muerta ceatest danger, however, 12 noc 
whether the psychologist will be wrong in some instances-- 
he will be--but whether he is willing to admit his error. 
No anno then Me ses cron will turn into little- -more 
than a lecture for the professional staff with the manage- 
ment representatives and other committee members serving 
e hUGtant audıcnee. 7Essentially,zthen, we are sug- 
festing that each psychologist must be willing to assume 
Mma titudo of Inquiry and participate in the search for 
the best possible assessment answers rather than attempting 
to promote his own viewpoints as being professionally 
Ole c T: 


The professional staff sets the tone in this regard. 
Consequently, if the professionals operate more as searchers 
than advocates, it 1S easier for the management representa- 
tives to also assume this approach. Actually, they are 
inclined to do so anyway and need only the support and 
encouragement that the psychologists! approach can provide. 


Thevelh we have indicated that the manggement repre- 
sentatives look to the professional staff for leadership, 
this 1S not to suggest that they are or ought to be 
passive followers. Indeed, our experience has been that 
the management representatives have been quite ready to 
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contribute their own ideas and have been quite perceptive 
concerning human behavior. In short, they have shown little 
fem@oncy toe be easaly cowed Or accepting of prima donnaish 
femmens On the part Gf the permanent staff. <Any efforts 
towards Tresoluti1iomwot diffemences have clearly been joint 
etrorts. Happily, the» toews has not beem} on "who is right" 
but Father on “whaieis right." 


Thus a difference in rating or recemmendation should be 
approached from the viewpoint that an agreement or con- 
sensus may be desirable, but not necessary or perhaps in 
Some instances not possible. Clearly, no one should change 
his opinions unless the available widence leads him to a 
different interpretations Most frequently, any disagreement 
ime hates Seoccunsmasmannesultofsattendimg to different 
portions of the information, and the difference is usually 
nessa by one or perhaps both of the disagreeing members 
acknowledging that he has not given the weight to a specific 
piece of information that the variable under consideration 
Warrants. 


when a difference does become apparent as- the committee 
members announce their ratings, each of the oar rering 
as 


views is given a hearing. The committee member who 

deviated furthest from the others is called on first by 

the Committee chairman. Theseenmittee members are instructed 
Porter this opportunity to explain their ratings as Wextier 
a Sales presentation nor a defense attorney's plea. Rather, 
each should present the evidence for his rating and the 
reasoning he used in interpreting the evidence. He may 


question the other committee members regarding whether they 
had considered the same evidence or if they can point up 
ataco la hisselogac. “Others mayassequest him to further 
APS evidence or Teasoning, or if it seems appro- 
priate, may assist him to better understand the meaning 

a thefraiiable 1n question or the Interpretation of the 
various procedures. As indicated earlier, any further 
CUsGueSicim that Moves amzene direction of persuasion Gr 
disparagement is discouraged. 


The chairman should also keep conversation relevant 
by sensing when a discussion has reached a point of dimin- 
Site returns or has begun to Stray too far afield in 
Geffense or rebuttal@@t a p@ant of view. In this context 
it is important that the professional resist the temptation 
to lecture to the group. We have already mentioned this 
in pointing out the danger of operating as though the task 
were to present his infallible judgments. Even though 
the professional staff member assumes the spirit of inquiry 
we have described, he is in some danger of operating too 
Ala instructor role, for frequently he sees an 
opportunity to enlighten the group concerning what he 
feels is an especially valuable psychological insight. 
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We do not suggest that he always refrain from following 
up such an inclanation, but ve do recommend that he do so 
with exceeding caution lest he sabotage his own efforts 
by becoming 1rrelevamt or technically overbearing. 


Betome turning to a consideration of the development 
of recommendations, we should mention that most of the 
disagreement in any integration session occurs during the 
rating of the variables rather than in the making of 
recommendations or the evaluation of potential. We feel 
that this is as it should be. The scales have demanded, 
in effect, that thc committee recognize different percep- 
Evens, discuss these differences openly, and clarify their 
thinking. By the time the developmental, placement, and 
other recommendations are solicited a clear and more 
unTrformsview of-thewsassessee generally prevails among the 
committee members. 


Placement Considerations 


In considering the various placements that might be 
Io tC Tor ae as sessces tie pı maryscriterion is 
that the placement be one that provides opportunity for 
hamsto rely heavy upon his areas of strength. It is 
rss Fosoph real -comvietieon thal Doth the individual 
and the corporation are best served by placing him in 
dar aS Maw en he 1S jostulikely LO experience SuUECEeSS. 
Common sense, our own experience with the program, and 
Pieced) Cvidence Supplied by Successful business leaders 
Sugeesertnat the probability of Success is greatest when 
theta tales tempo ame os trenes rather than) trying 
to respond to the "challenge! of positions requiring hin 
to surmount personal liabilities. Consequently, any 
Other considerations become relevant only if this primary 
CINE On 1s met. 


However, the committee can only provide suggestions 
Silec a number Of factors Other chan the individual's 
STiensehsgcre also conslderedón praecement planning. 
Among these are present corporate needs, the pattern of 
future corporate growth, availability of appropriate 
supervisory climate, and the broadening exposure provided 
bym@the assignment. 


Although the review of the scale ratings has pointed 
Up ences committee S Consensual perception of the assessee, 
o hat the first step in othe consideration of 
placement opportunities be to gain agreement concerning 
which of the personal qualities should be capitalized 
Duo sera lily, this cntalls only a restatement of 
ios sees postive qualities in job-related terms. 
For example, assume the individual is organized in his 
Vora ner ogeh, has dualytical skill and extensive finan- 
Slebeeteiming, but has limited forcefulness in interpersonal 


12122 





Si aren results in little impact on others. A posi- 
Bor eguigine integration of complex financial data but 
Pau@eielttle or no responsibility for supervision or "selling" 
Of reports would most likely provide best utilization and 
Mex imum SatiwStactien tor this individual. O£ course, if a 
position could be found that also provides an opportunity 
Pain PreatcYr GONricence In interpersonal areas through 
being permitted to make well-prepared reports to a group 
which would respect the assessee's "expertise" and bear 
with him in his fumbling but courageous speaking efforts, 
So Mmbueh the better. However, this would be a secondary 
Deeewent target, 


Once the personal strengths to be utilized have been 
elle :iv spelled out, the committee turns to the task of 
naming several positions in the company that meet these 
reqmarememts. These positions are listedemot as reconm- 
memded placements per se, but rather aS illustrative of 
Bier type Of Opportunity that shoulda be Sought tor the 
Ee S a O eh examples help clarity for laver users of 
the information the committee's practical interpretation 
of all of the rated qualities. It is desirable to identify 
Peeitions 1p several diftierent divusions if at all possible. 
Usually, however, the three management representatives on 
mene committee at a particulametime are not very familiar 
with all segments of the company. This further supports 
the concept of considering the committee's placement 
suggestions as illustrative only rather than as compre- 
hensive. 


This is a part of the assessment process in which the 
background of the management representatives is especially 
Poise. they arewasked to draw upon their knowledge of 
the different phases of the business to identify positions 
that they feel make the type of demands the assesseec's 
native ability, skills, style, and experience have prepared 
Rim to meet effectively. These positions are carefully 
considered by the committee with the professional staff 
representatives usually taking the role of the devil's 
mevocate, it 1S at this point that disapreement Is most 
OO ecun during tne recommendation preparation phase. 
It is not unusual for the management representatives to 
mean slightly different responsibilities by use of the 
Same term. Areas of diSapreement are resolved, or attempts 
are made to resolve them, in much the same manner followed 
during the scale evaluation. If the ambiguities of a 
panties position termimelogy cammot be clarified, however, 
emeomposttion should mot contribute to the subsequent 
Somemtaven o1 Others by being used aS an example. 


‘Occasionally it is felt by an assessment committee 


that no current corporate positions actually are illustra- 
tive ofhow best to employ the skills of a particularly 


12185 








unique assessec. In these instances it might be helpful to 
Suggest the design of a new position and the way in which 
it would function. 


Also, it may prove constructive in some cases to provide 
eales or positions that definitely should be avoided 
because the likclihood of failure would be quite great. 

In fact, in a few instances, we have evaluated individuals 
fOr whom it secmed less important that specific strengths 
be utilized than that some very pronounced weaknesses not 
Io ca especially for these assessees, it is quite 
alo to take time to delineate the types of positions 
compe avoided . . 


Developmental Needs 


When introducing the area of development, a good ground- 
work is laid by subscribing to the "self-development" theory 
meee onomc devemops a person bet the person himself. Cor- 
porate management is responsible, however, for providing 
Pe Dendsopportunitıes that will enable the individual, 
ip wie™ SOnCh@Oscs, to cOntinue lis development in the arcas 
mec Which his experience and initiative have already led 
ACOSO wat pes ol opportunities fhe corpor- 
OSO dp men ide, wirt 15 Well to"turther point out that 
me salıtıesgerhat anmewalneady strengths should be further 
refined if possible. Weaknesses should be considered only 
te tent geet they are significant impediments tə kıs 
performance. ixCessive attention devoted to those liabili- 
Aaa tao not seriously limit the use of his strengths, 
but rather detract in a more-or-less academic way from his 
being a total person, may not be either a constructive or 
leg timate foews of the corp@ration. Therefore, i» evalu- 
wate cthem@esirability of public speaking training, for 
example, to the extent that such cxposure would increase 
ready moderate-to-higeh level of competence or is a 
necessary skill to function effectively in positions 
realistically seen as options for the assesseé, it should 
be encouraged. However, it is highly questionable that 
public speaking training should be recommended and sup- 
a ai he corporation if it might not prove effective 
or simply because it might assist the individual in gaining 
an additional skill when in reality it would be unlikely 
oooi additional competence would matcrially improve 
his likely contribution to the company. 


We belicve that the best development the corporation 
can provide comes from job placement that gives opportunity 
tov amply Strengths. Employing strengths increases the 
likelihood of successful accomplishments which in turn 
io wemes wecontidences and oftem providcs greater insights 
into needs for broadening and development. lIt is upon 
Mia er that other devclopmental efforts can bulld and 
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Prove eltective. Consequently, some of the most important 
developmental recommendations are contained within the 
Section on placement considerations. 


In evaluating further developmental needs beyond those 
Sacisited by proper j@b placement, the committee can be 
Nen a Istine of available formal and informal learning 
opportunities. These might include all of the current 
training programs conducted within the company and also 
examples of external sources and activities which may be 
ip ropriate. It 1S not unusual for the econmitcee to 
decide that special developmental opportunities beyond 
those contained within the recommendations on placement 
and supervisory climate are either not required or perhaps 
not Warranted in relationship to probable return on the 
ollas invested il) providine the individual with a partic- 
ular opportunity. Return on investment is not an easy 
concept to measure when operating in the often intangible 
area of personal development. Nevertheless, it is not 
nout merit as a guiding principle. 


A wide variety of specific recommendations may be 
offered. The diversity may run from reconmendations of 
aeteave Of absence with pay to permit completionof a 
doctoral program for a talented researcher to a bibliography 
of suggested readings in management practices for a young 
Seem isor. the oWerriding concern is that tl@™recommenda- 
tion will enable the individual to be more productive for 
the company if he chooses to take advantage of the suggested 
Spportunity. Here again, however, the recommendation of 
the committee should be scen as suggestive since many 
factors and conditions beyond their knowledge and control 
bear on the advisability and timing of specific develop- 
mental actions by Or on behalf of the assessce. 


Potential 


The final judgment made by the assessment committee of 
any assessee is an evaluation of his potential for contr1- 
Mies” to the company. Again, tor this rating a 5-point 
de Mea hali- step intervalseis employed. Opportunities 
(numbers of jobs) at higher levels of management diminish 
quite markedly as one moves upward. Relatively few indi- 
viduals@@ver move into Serious contention for officer 
eve | Swesaiitions. And yet nearly all higher level positions 
are iWed by individuals who move from the levels of those 
now being assessed. The committee, therefore, should try 
to estimate the maximum level of contribution as a general 
guide to the company as to its resource potential for 
Mino the top levels ok responsibility. In actual prac- 
tice, assessment committce judgments of potential are dis- 
mabata en a normal curve basis. 
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Pectin mweiic rating on potential, the objective is to 
estimate how far in the organization the assessee should 
advance and still make a constructive contribution bcfore 
the normal retirement age. This may be different from the 
level to which he will actually advance. The rating on 
ren cialemay be higher or lower than the level he will in 
reality attain. Undoubtedly, some men will be advanced 
BBetche Or@amization beyond the point at Which their abili- 
ties enable them to be effective. Others, through circum- 
stances not completely within their control, will not 
acia cs ito responsibility levels they could competently 
Bandle, It is a major objective of the assessment program 
to provide evidence and judgments that will improve the 
mace ilinood of each individual's being given the opportunity 
eom cach his maximun evel” ofvcomerirpoucion. 


The assessment program will not be without error, 
mewever,  lhewWemis sumprrsingily little disagreement at this 
point in the evaluation, büt consensus is not necessarily 
equated with accuracy. We simply trust that the careful 
deliberations of a sincere committee will provide general 
guidance that can be constructively üsed in. subsequent 
manpower deliberations. 


After covering current operational methods of 
Nana tion, 1t as considered prudent to review some theory 
on how the process might be improved. Taft [61] provides 
en3s function, writing as follows: 


Let us conclude this section on the safety in numbers 
assumption with a proposal to combine the advantages of 
both multiple techniques and multiple assessors. The 
suggestion is that each assessor be given a limited amount 
of information on which to base his assessment judgments 
about the candidates, each assessor to receive different 
information. The assessments will then be pooled arith- 
metically. The information supplied may be objective 
or subjective, atomistic or molar, and may range from one 
item to life-history, or a test result, to a projective 
test protocol, an interview or the observation of behavior 
mM aemintvacire Situation. This prosedure would enable a 
Vast amount of data to be integrated without problems of 
weighting since unit weights for each assessor's contri- 
bution would be adequate--this would be analogous to an 
inventory that gives unit weight to each item. With 
adequate organization of the assessment program, this 
would permit several assessors to contribute to the final 
assessment so that different viewnoints and personality 
theories can be represented. This approach seems to be 
at least worth experimenting with. 
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Even if it is found that increased numbers of assessors 
increases (perceptibly) the accuracy of the assessments, 
there 1s still a fine calculus of cost in human time and 
effort to be computed. The decision to augment the panel 
with additional assessors 1s a function, among other things, 
of the gradient of diminishing returns, the ability of 
evagrable extra assessors, the cost of using them, the 
effects on the candidates, and the desire to allow execu- 
tives in the institution to participate in the assessment. 
The proposal made above of having many assessors, who 
contribute small pieces of information, may make it possible 
to conduct multiple assessments comparatively cheaply. 


In summary, Taft makes the following recommendations: 


Each assessor should be given no more than two or three 
units of information; there should be a large number of 
assessors whose predictions are pooled arithmetically, end 
without discussion. 


In selection assessments, if committee decisions are 
desired, the assessors who are particularly well-experienced 
in the criterion situation should be given special influence 
IO nante tinal decisions, provided they have uheen 
Showneto possess good ability to judge persons. 


k. Feedback of Results to Candidates 
Once again, Byhams [11] treatment provides a 
brief overview of the topic. He writes: 


Metro mece most important, yet most hazardous, aspects 
GE assessment center operation is feeding the reports back 
to the candidates. Companies handle this in widely dif- 
ferent ways, depending on the purpose of their centers. 
iar ecencommanles ortcr candidates the option of receiving 
or not receiving feedback. Between 60% and 90% ask for 
it. These companies find that candidates who do very 
wel] and those who do very poorly usually know where they 
stand and do not request feedback, whereas those in the 
middle want to find out how they did and get hints for 
self-improvement. Some companies give feedback to all 
candidates automatically. 


ine almest all eases, andecertainly in companiesethat 
are strongly concerned about management development, results 
are carefully couched in terms of the directions that a 
candidate's personal development should take in the future. 
The candidate's impact on his fellow candidates may he 
communicated to him to make him more objective about himself. 
His performance on individual tasks may be discussed with 
an eye to establishing a plan to overcome noted deficiencies. 
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When assessment and training are combined, it is.pos- 
“ame etOeproviage some feedback to candidates prior to their 
teavime che center. In some Companies, a candidate must 
wait weeks for a feedback interview. Obviously, the sooner 
the feedback interview takes place, the more impact the 
training and development recommendations will have. 


If a psychologist is available, he usually has the 
responsibility of discussing the center's result with the 
candidate. Otherwise, assessors or former assessors are 
menthe Tespos bility: 


meU r on i de ethe tolbowing collinselinga 
murdance to their assessors: 


1. The following is an outline provided to facilitate 
counseling feedback based on assessment exercises. The 
Beer ctıon, collation and feedback of the information will 
maker place as follows: 


pew ineztonmputren Group wiki previde each counsélor a 
Macke of information relative to the individual he is 
assigned to (counsel). This packet will contain a computer 
primtout of all rating scales arranged by dimension, and 
the @arrative portion frem each exercise. A summary sheet 
containing information which relates to each assessee 
aerived 110m paper and pencil tests will also be included 
in the packet. 


B. The information given to the assessee will center 
around the dimensions . . . (specified). The counselor 
1s charged with assimilating the information from the 
various exercises into the dimensions for counseling 
feedback. j 


C. The counselor will prepare his counseling inter- 
view in the following steps: 


SEP ONE: Read the narrative portions from each 
exercise tomgetsan overall impression of the assesseée. 


SIEP TWO; Study the computer printout and paper and 
pencil test results. 


STIER STEREE: Prepare a Narrative Report describing the 
aaa lo lbs. repontawill follow the dimensions .. . 
(listed). The report will contain all relevant information 
Monica rratives, thescomputer printout, and the paper 
ama pencil fest summary sheets. This report will serve 
Bowtiembeasis tor the counseling interview and for ARI two 
Pear roll ow-up. 
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SILO” Prepare the counseling interview. The 
Sequence of dimensions covered should meet the following 
ETA: 


a. ime first dimension selected should be the strongest 
of the assee's relative abilities. 


b. The initial dimensions should be the easiest to 
present from the perspective of action the assessee can 
take to address his weaknesses. 


C. The dimensions in which the assessee is weakest 
should be in the middle of the sequence. 


d. A dimension in which the assessee did relatively 
well should be reserved to conclude the interview. 


STEP FIVE: Review and select tapes to support the 
interview. 


(Notes: (1) The 14 leadership dimensions assessed are: 
Social Skills (Interpersonal Competence), Communications 
Skills, Adaptability, Motivation, Forcefulness, Mental 
Ability, Decision Making, Administrative Skills, Organi- 
sana Ll Identification, Effectiveness in Organizational 
Leadership Role, Supervisory Skills, Physical Competence, 
Wechnical and Tactical Competence and Problem Solving 
Ability. 

(2) Step Five refers to the preparation of a 
ar Mimute Videotape which the counselor records. fromepie 
MmraeOtapes Of the leaderless group discussion and the can- 
edate Ts interview of another role-playing candidate. 
Mrect Views with the USAIS assessors disclosed that the 
edited videotapes were a powerful counseling tool and were 
often able to dramatically convince candidates who had 
doubted the verbal feedback on certain specific points.) 


Finkle and Jones [85] again provide a thorough, 


Peeenouch specific, description. They detail the feedback 


procedures employed in the Sohio assessment program which are 


Somewhat atypical in that they use pscyhologists to conduct 


the feedback sessions, where most programs utilize lay 


Mecesscors., In Spite of this, the description contains a 


great deal of general information which is applicable to 


may feedback system. They write: 
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Pee baste problems quickly arose when considering 
the feasibility of a feedback session . . . 


The first of these problems concerns the cost of the 
feedback in both time and money. The other, and even more 
important, relates to the possible misinterpretation of 
the results by the assessee and the consequent damage to 
confidence and desire that may ensue. 


With full realization of the problems involved, we 
decided to follow through with a feedback session for any 
assessees who were interested in requesting one. Weighed 
in our deliberations were the following considerations. 

The cost is not great relative to the investment already 

made in the program, “Yt has Deen our experience that an 
assessment feedback requires at the maximum an hour and a 
half time and more typically forty-five minutes to an hour. 
When assessees operate in areas remote from the home office 
location of the psychologist, it is often possible to arrange 
a meeting when the psychologist is at a nearby company 
location for other purposes or when the assessee has occasion 
to visit the home office on other business. 


Furthermore, the cost must be weigheg against more 
intangible outcomes. It is impossible to calculate the 
amount of resentment toward the company that might develop 
in some assessees if they felt that pertinent information 
concerns tel personal qualities wes being deliberately 
withheld from them by management. The loss of a well- 
qualified individual as a resuit of such antagonism would 
be a real loss to the company. Also, the cost question 
must be evaluated against the possible gains in personal 
growth that the feedback might foster. We will return to 
Pure ue a little later. 


Inesnoserblemeoative erfeets resulting fromemisin- 
terpretation of test findings or the indelicate handling 
anche veportıne of the findings to the assessee can 
adversely affect an assessee and the usefulness of the 
program. Indeed, even though the feedback is conducted 
by a psychologist, the likelihood of misinterpretation or 
improper handling is only lessened--not eliminated. Yet 
there is no assurance that distortion and misinterpretation 
do not alse arise in the absence of a feedback. Obviously, 
each of the assessees speculates, at least privately, about 
his own abilities vis a vis the other participants during 
the various group exercises in which he is involved. How 
realistic these self-appraisals are depends upon a number 
of things, including the maturity and insight of the 
assessee, his previous experience in similar situations, 
avemiiceconriaence in his capabilities. 
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Without the opportunity to test his*own interpretations 
against those of the assessment staff, the assessee may well 
cer an the alrectionsoí being a bit too optimistic or 
pessimistic concerning his future prospects with the company. 
In extreme cases he could develop a totally unrealistic 
outlook regarding his potential for advancement. Thus the 
feedback should further the ability of the assessee to better 
analyze and interpret his performanee aifd the possible 
rk arıons or tııs relative toas cafeer aspirations. 


The goals of an effective feedback therefore are twofold: 
(1) to provide the assessee with an opportunity to compare 
seli=emwa la tionsolt his pertormamee with the interpretation 
of the assessment staff and (2) to provide him with the 
appropriate amount of feedback information in a growth- 
oriented context that makes the information as acceptable 
and valuable to him as possible. 


Initiating Self-Development 


Ihe ma om ponon Ot the writings of psychologists and 
PsychilatriS tse Ssupmerts the view that the starting point in 
ame, seli-developiental effort is a willingness on the part 
of the individual to examine himself and determine his 
current abilities as carefully as possible as a foundation 
for formulating personal goals involving change. To be 
Sc, fueras Opel sonal WorOwed Js sc best difficult 
to govern and achieve. For in addition to a motivation 
Besbetter fulfılı his potential Capability, an individual 
needs æa means of seeing himself as clearly as possible as 
he is. et ue loge the assessment program offers assistance 
1m) thiseeTespeet in at least two major ways. 


PS e tere IS Considerable Oppo Unity during the 
Coursememethe three days of assessment to critically examine 
ame ln the measures of intellectual ability, te 
assessee comes to grips with the fact that he can handle 
some types of intellectual tasks more comfortably than 
Other types. Many of the attitude and personality inventory 
ear Nito examine Lis sbelmers and his personal 
Slew eso, the perseñal artery iem confronts the assessee 
Ween tren question otr persenaie coca lseand his evaluatiron of 
his own strengths and liabilities. In the group exercises 
the assessee has the opportunity to see how other young 
BUSsHessinenm tackle problems and conduet themselves in a 
COMbecrbirve Climate. “Finally, when conpleting *the final 
SOCTOMCTIIC sguiestionmaire at Ihe conclusion of the program, 
he becomes very much aware that not only is he evaluating 
CulLers Me werms of their type of personal impact, but that 
they also are evaluating him. Naturally, he wonders how 
he impressed the others during the three days of formal 
and informal interaction. 
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Poreiiany Ot Uinemcemployees this is the first time that 
they have really stopped and taken a look at themselves 
in depth. Prior examinations have typically focused on 
MieeorrectWal characteristics only. From various college 
quizzes and aptitude tests, they have gained some insight 
concerning their standing relative to general population 


Mmewstuaent norms. There nay have been some vocatiomal or 
edueatseonal eaunscelhame at the time of the selection of a 
galesc and earcer. Withetew exceptions, however, they have 


had little occasion to pull back and reflect upon themselves 
and viduals in a broad sense. 


The second means by which the assessee receives assis- 
Daneemineseltedeve le~ment comes in his opportunity to receive 
S e pue: concerning the Interpretation of Ms performance 
and a Chance to discuss himself and his future with a member 
gr the assessment staff. Such a discussion, if well handled, 
can become the basis for more sound career planning and 
Muerde both a clearer focus and direction to personal 
developmental efforts. In short, the feedback session 
should further the objective of personal growth’ for those 
who are so motivated. 


Be sona le Peedback--Not Performance Appraisal 


Because a performance appraisal interview and an assess- 
Maure pack both deal WEL An individual's behavior in e 
work situation, we can further understand the nature of the 
assessment feedback by comparing the two. If the assessee 
has had performance reviews concerning his accomplishments 
on the job at fairly regular intervals, he will expect the 
assessment feedback session to be rather similar. However, 
Me snoot the Case for the following reasons. 


Pao veaiely , stieres!s a pronoun@eau difference in "le 
eee etve OF thes wO sessions. dhe goal of a performance 
appraisal discussion is to provide the individual with 
knowledge about how his specific job performance has been 
regarded for a stated period of time, usually one year. 

By contrast, the assessment feedback deals heavily with 
the effects that personal characteristics have on the 
assessee's present and potential capacity to contribute 
productively in a business setting. 


teerseuicgely meld that performance appraisal review 
Er ons chawe tie” most constructive effect when the 
discussion centers about the accomplishment or lack of 
progress on agreed-upon work objectives. Discussion can 
tien pProceed to a consideration of the implications this 
Br sformence has tor the attainment of future work objectives 
ete remmomthic dividual s current job. Tlowever, when 
the appraisal session moves away from the focus on 
objective results into an examination of the underlying 
behavioral characteristics that made the performance 
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PSSS the G@erensiveness of the subordinate increases as 
e ability of the Supervisor to evaluate this area effec- 
Eivetly Comes into question. Some discussion of behavioral 
aee Stee 15 perhaps unavoidable, but the temptation 
to revamp the subordinate as a person is difficult to resist 
and there is danger that the session will digress noncon- 
Sieuctively EromPene task of providing knowledge concerning 
job progress and future work plans. 


contras ele assessment tecdback 1s less effective 
fo the extent that it becomes Too related to specific job 
performance. [Illustrative examples may be gleaned from 
the assessee's current job performance, but the emphasis 
iecleariy upon the person as a contributor to the organi- 
zation in general rather than upon his behavior in a given 
assignment. 


Another factor differentiating the feedback from the 
Per ormmance appidisal is simply who conducts it. This 
eean aa eS n onae diflferent- obDjectives and the 
relative appropriateness of the manager to deal with the 
or mataon to be Gevered. Most employees accept the 
concept that their manager is a legitimate authority to 
evaluate the results of performance in relation to agreed- 
upon job objectives and responsibilities. Far fewer regard 
their managers as holding the prerogative to either assess 
Or attempt te change basic behavioral characteristics which 
rent imencestine atemawent Of work results. Therefore, 
when the manager moves into these areas, the subordinate's 
detensiveness increases. 


Bere tieareverSels strue: Lor psychologists. 
Many question the capacity of the pscyhologist as an evalu- 
IO Bulle extent of accomplishment of specific job goals. 
ere many probably agree that aS a result of training 
Ae ne iencs thesps, Cholopist 1s likely to have sug- 
ee stıons concerning change in personal qualities that 
might yield dividends in improved personal satisfaction 
and accomplishment. 


Also the psychologist stamals in a different power 
relationship to the assessee than the manager. Although 
Fiesms»choloeist may be recognized as a representative 
of the company, there is little perception of the psycholo- 
gist as holding any immediate real power over the assessee's 
fúture as is the case with his superior. The impact of 
power in a relationship 1S such that there is an increase 
in defensiveness and a decrease in exploratory and creative 
thinking. Thus even though a manager may have a positive 
superior-subordinate relationship with the assessec, the 
inherent power structure tends to have a dampening effect 
upon the amount of personalized information that might be 
exchanged. Basically, then, both the perceived legitimacy 
of the psychologist to offer suggestions in the area of 
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“personal growth and the neutral power relationship increase 
the likelihood that the assessment feedback will become a 
session in which there is a meaningful focus upon personal 
qualities and formulation of growth-enhancing plans. 


Mm summary, then, the @ifferences between appraisal 
Meedback and assessment feedback in objectives and personnel 
involved should be clear to those involved. However, these 
two activities do have a complementary relationship to one 
another. Both activities are directed toward improving 
performance and providing guidelines for further develop- 
Node ali, Dot iS vetas vehicles for the individual 
to evaluate better his current performance and perhaps 
gain further insight as to how he can capitalize most 
BESUCH UNO Ce "ST ronger aspects of his performance 
and adjust other attributes if his capabilities and back- 
ground suggest that this is a feasible alternative. 


ereraa net e manager and the psychologist Doth work 
o TOP and or utze better ithe individual personnel 
resources of the company. The manager's emphasis in 
Poerkine with the individual is on the "what" and "how" 
Ms cr OL Nis results, At the same time the psychologist 
Som e Lp Ate individual better understand the "why" of 
his behavior and gain a better understanding of "where" 
Me apt to find satisfying application of his talents. 


arlo he Feedback Session 


Let us turn to the mechanics of the feedback and two 
important questions: How should the session begin? How 
much information should be conveyed? These questions can 
be best handled if we keep constantly before us the pre- 

e ae discussion and remember the objectives of the feed- 
eee in ectual prectice the conduct ot the feedback 
session created several problems for us early inthe program. 
heuagm@ees portion Of these problems ¿rose from the fact that 
ete tent attentionmiwas paid Lo the context of the 
session. 


Forza anple, onr eariyareedback Sessions were too much 
Bere en deut) arone Way report back to the assesseegof 
Moyet ed=sceocnent findings. The Criterion for a successful 
Beedibdeweriis became that of full coverage of information, 
Mei terion was Not articulated and explicit. 
Consequently, assessees were sometimes given information 
aa perhaps interesting but of questionable relevance 
perception Of their career objectives and 
personal needs. Had we been more fully cognizant of the 
full role the feedback session can play, we would undoubtedly 
Ia uicker to see the joint problem-solving nature 
et the feedback session. 
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Another ditficulty of our own making was to approach 
the feedback task with the rather naive assumption that 
all assessees aspire to positions in top management. The 
Peeabach in tormmation was presented to the assessee in the 
f[oweeective ot” the assets necessary to progress to top 
Inca AS a result we were often placed in a position 
of indicating to someone who had modest aspirations in the 
Minot taco that 2S prospects for advancement to positions 
of heavy responsibility were quite slim because of the lack 
Pere levant personal qualities. With the advantage of 
ASE ION recognize that the position we had werked 
oW elves Into was not unlike that of a talent scout for 
MS To pata OperaMtelling am average chur@h choir 
memectrethat heehad littke prospect ofesuccess as the male 
Ice Met. Even though the choir member had no 
maae abti ons, he would be a bit chagrined at the 
appraisal of the scout. Likewise, our assessees who had 
IE Re aspıration nor ability for top management 
responsibility were nevertheless discouraged and resentful 
On receiving information couched in a top management frame 
Gieene ference. 


Tees Carly foedhback sessions, it was particularly 
painful to anticipate and conduct a feedback with someone 
who had performed poorly throughout the assessment session. 
Buben we Changed the feedback from a simple 
NO results to a probiem-soiving efiort, even 
these sessions became managcable and productive. 


Fowtunately, even those who had done poorly in the 
assessment session either had an awareness of this and 
had adjusted their aspiration level accordingly or did 
Ina run cealistically high ambitions initially. We 
have found that when an individual is given an opportunity 
momeesceribc what if 1S that he wants from his business 
VETE” men express aspirations higher than the 
eee eit stati felt they were likely to attain. Once 
the assessee outlines his objectives, it becomes possible 
[er Erreshas pertormance to these objectives. Also, by 
Ie rvery active involvement from ham early in the 
session, 1t is easier to maintain the session as a joint 
A an tO try to introduce artificially a two- 
way information exchange such as had characterized our 
Caviyecr torts. 


Quite often an assessee states that he wants to progress 
foes ito eabilitmes Wall permit, which is perhaps a 
subtle way of asking for the assessment staff's rating of 
No eL tral. iin discussing potential it is important to 
Stress that his potential depends upon his own talents and 
Sis. upon his exposure to jobs and people in order to 
best use and prove his abilities, and upon environmental 
conditions, including the need for his talents, the 
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Ay O: Similar talent, and a grain or two of good 
Ilave found 1t most effective to concentrate on 
talking about the relative strengths and weaknesses he has 
aos sane. in general, how he can continue to grow 

in effectiveness as a person. 


Howeroechn Information 


The question of how much information to provide is also 
astchorny issue which is best resolved in each specific case 
Ware Wome walert to the counseling context of the feedback 
and the stated goals of the assessee. The assessment findings 
Should be evaluated and integrated in such a way as to meet 
Giemdsseases S needs mather than simply be presented in a 
Mette ee leare it fashion without such integration and 
cen etario. Little 15 gained by giving him the "straight 
Poo si remoOuUL dn attempt to relate it to his individual 
Gesires and experience. Specific performance on a given 
test or exercise may be used to illustrate a broader char- 
SIC Due tie overriding judgments of the committee 
Peete principal Subject for review. 


Overall, it is valuable to keep the discussion focused 
on a descriptive level which is more integrative than 


yee e ror example, “You seem not to take full adran- 
tage of opportunities to tune in to the needs of others in 
e and action. ' Inierence is made without cor: 


demnation. An idea is offered for discussion and as much 
as possible the session is oriented about what the assessee 
Ica and Willing to accept. 


aso the session 1s directed toward description 
Ae pretation Im the interest of self-understanding, 
Piero less likelihood of distortion and difficulty. Nor 
HSethere danger in discussing the implications of present 
Acs tcs for the type of person the assessee hopes 
to become and possible steps to take or questions to ask 
himself as he moves toward his objectives. However, danger 
Sin trying to discuss areas that are essentially 
beyond the control and management of the assessee himself 
or the psychologist with whom he is working. 


One of these areas is the question of long-range potential 
as evaluated by the assessment staff. We have already 
expressed our feeling that the information can be most often 
misinterpreted because it is abstract, future-oriented, and 
subject to so many variables outside the assessee himself. 
Yet one of the basic questions the assessee wants answered 
when he requests a feedback is how much potential for 
advancement he is regarded as having. His question probably 
pects om a need for the security of reassurance that his 
fate is not sealed and his valuc to the company short-lived. 
Actually, this reassurance can be given to most assessces 
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without the necessity of delving with great precision into 
IS Ma cero potential. fhe selection criteria for the 
program are such that few if any problem cases are evaluated. 
“Therefore 1t 1s almost always possible to say to the assessee 
A ace ricy that he las potential for growth within the 
Ma tion. BIS fact, combimeo with the general realasm 
of the expressed objectives by the assessec, places the 
question of potential in proper perspective. Most important, 
no assessee should be left with the feeling that a ceiling 
has been established for him. He should recognize that any 
cane or potential Ys Just that--an estimate. Wany Cir- 
ermetanee-, LOremost OL Which is The assessee's own appli- 
Cautemeand development of his abillties, can atfect his 
potential. - 


Morona. tire mreneral ques trono potential, there are 
Stier tations that it may be desirable to place upon 


reporting the data to the assessee. Thus specific jobs as 
opposed to job families should not be discussed with the 
Media as possible future placements. It should be 


er ed ta a eee job titles are developed in the 
mtv ation session samply to exemplify better the type 

TE OTON aoul capitalize upon the assessee!s 
strengths. These are examples only. When discussed with 
the assessee, special care must be taken to make sure the 
individual understands the illustrative use of the position. 
Otherwise, the individual ts apt to bepin thinking In terms 
ot a specific job and be disappointed if it is not soon 

O ered even thouchn a future proffered posıtıon may actully 
Meets needs and the company’s needs more ecrrectives , 


Likewise, to the extent that the areas of supervisory 
Slaiiete wana traaving are discussed, 1f is desirable to 
den eiess only those thıngs over which the assessee has 
control. im speanine of has relationships with authority, 
AU to ata amas css 100 0: his Immediate superior 
unless the assessee brings up the topic. Frequently, the 
Poy enolGeist conducta =ememirecdnack has information con- 
cerning the superior. Discreet use of this knowledge may 
be helpful in aiding the assessee in his relationship with 
the superior, but most often the psychologist will do well 
pOmuchiomner that thoro mOi the feedback is-the assessee-- 


notcshis supervisor. Frequently, the assessment staff 
recommends outside advanced management programs as company- 
Sponsored training activities for the assessee. It may be 


useful to advise the assessee to keep himself open and 
receptive to training opportunities his management provides, 
but beyond this there is nothing to be gained by discussing 
specific programs where the assessee does not control the 
Selection. However, any relevant activities the assessee 
can initiate fully on his own should be openly and fully 
poe de, 
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a E Oral or Written Report 


Wee cedmdckesessions have been completcly oral. This 
taismprovicdecmeus with desirable flexibility and has permitted 
a full “back-and-forth” discussion with each assessee to 
assure better understanding. However, cven with oral feed- 
backs there is the danger that misperceptions may arise and 
go completely unnoticed by the psychologists. There is also 
the disadvantage (at least in the eyes of the asscssee) 
that he has nothing tangible to carry away with him from 
Elem se Sea on. 


Though we have not employed a written report for assessees 
aumSehao,#suchean approach could of course be considered. 
Presumably it should be descriptive in nature and similar 
to the one provided for management. 


An extenston of the arguments favoring a written report 
for the asscssee would lcad to the conviction that he should 
A be eiven.a copy of the report prepared for management. 
Another view might be that he should be piven all detailed 
mieeonmlatioOmean terms Of test scores, interview reports, 
ems e teporis, and SO On that were used by the Integration 
eemmittce In torming 1ts judegméents. 


Cora a DPen te In establicnimno tie most etfcCeuumemmeccdiack 
COMO AS veen to maximize understanding on tho part or 
the assessee as to what has been concluded about him, how 
Ae uscd, aid low it 1s likely to affecti his Car er 
with the company. We doubt that an open presentation (even 
Morandi or Tay data best serves this purpose in the 
Mor the Gemplcxities of the techniques and exposures 
Meee to obtain the data and in view of the nature of the 
ate judements contributing to ihe conclusions. A 
aten Teport may be justified Duc not if it cuts off 
discussion that gives the psychologist himself feedback as 
to the progressive understanding of the assessee in the 
feedback process. Oral feedback, when well handled, seems 
tie best job of Gevelopine full and accurate under- 
Standing. 


It should be remembered, however, that we are secking 
to provide the assessee with an understanding of the signii- 
ance of the information for his career as well as of the 
contento the amGornation ltvself£. Perhaps the best indica- 
As understanding isa reading by the psychologist 
of tle assesscee's emotional reaction to the feedback session. 
Signs of disappointment may often indicate a misunderstanding 
Seetoreotetnc Information or its Significance. Tor persons 
Wo Haye an excessively unrealistically high level of 
aspiration, discouragement is to be expected. For most 
individuals, however, the overriding reaction to the feedback 
Mas tructively accepted. Our experience suggests that 
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Br Ener son: MOct lakely to feel disappointment are neither 
Prasewwith exceptionally great talent and potential nor those 
with relatively modest or narrow abilities but, rather, those 
tath skills, abdiliíties, and interests that are about average 
Or aelitembe better thadlmavemagc for the population of 
individuals recommended for the program. This is, perhaps, 
somewhat like the reactions of "B"-level students in school. 
fie so tpertenceenecmnen sence mciact of the “C-level. student, 
io may have had some doubts about passing, nor the satis- 
faction of the "A'"-student who, though he may have expected 
it, as rewarded with the highest possible mark. 


Mae are several ways to help this “level” of person 
achieve an appropriate understanding of the assessment 
intormaiston and 1ts signi figcemee for Bin. The psychologist 
Nut remenber that tie restltseeé thie program confirm that 
the assessce is exceptional in ability and potential. Those 
selected by management to attend were seen as more talented 
Sn crearte o tentrial than their peers in the 
Dorkssettinp. Thewassessecs we arc now discussing were 
seen by the assessment committee as having as much or more 
talent andwporential than the typical, well-recommended 
assessee. Clearly, the psychologist should focus on the 
oa pe paa esca ability rather than on the point 
that some have been seen as having even more talent and 
potential ethan the assessce beim addressed. beyond this, 
owes Io LOmboatecrale whit Should have beenstold 
Elie dssessce Pidor to ule program and what Certainly nas 
been pointed out more than once at the program, namely, 
Clee meie lr LlirymemmeGctive is to establish and reflect on 
Als "menes oso. theta thirouen his greater 
awareness of these strengths and the company's greater 
awareness of them, they can be more quickly and more 
appropriately applied. Finally, it should be emphasized 
that an individual's potential depends upon opportunity, 
exposure, pertesmentce, and upon the judgement of one or 
mere key people who have the authority to affect the 
Midi Viattiaes Pron Tess... lie assesslient program operates 
PIE to improve respestire. The contribution the 
individual makes in the eyes of those he is working for 
and the opportunity those persons have to increase his 
wE Pons bility arc still the primary determiners Of his 
Caweer adwamcement. Each person's future depends upon the 
extent to which somcone who feels that person can make a 
Stron er contribution in the organization is insa position 
to loss emer hin about ıt. 


Meca msc the eritical importance of properly 
designed and exccuted feedback sessions, it is considered 
important to close this section with some additional words 


Ion Concerning possibl@ dumaging cffects which can 
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aap [en 


Pale “Meyer et. al. [49] report the following #indinmgs 
from a study done at General Electric: 


The average G. E. employee's self-estimate of perform- 
Pe@eembelore appraisal placed him at the 77 percentile. 


‘Only 2 out of 92 participants in the study estimated 
their performance to be beTow average. 


Commenting further on these findings, Thompson and Dalton 
[63] write: 
a ihe probability is, therefore, Chat for 70% to 80% 
of all technical personnel a comparative ranking would 
be a deflating experience. Certainly, all would feel 
criticized, and those who asked for the reasons for their 
unexpectedly low rating would receive more specific 
See casn. Wudeingesnom the 6. E. findings, this Would 
Mame a disruptive Citect on the performance of Most of 
them. In addition, it would have a negative effect on 
Pica sl eS ca 
The message voiced repeatedly in these, and other, 
articles is that feedback of results is one of the more 
difficult and crucial factors in the design of an assessment 
program. Thompson and Dalton [63] express it well: 
Po hEOomlanee appraisal -Louw@hics onone of the most emotional ly 
charged activities in business life--the assessment of a 
man's contribution and ability. The signals he receives 
about this assessment have a strong impact on his self- 
esteem and on his subsequent performance. 
Parse tlc COlmente CONCeCINed@epersOrmance appraisal, this 
writer considers it to be equally apprřicable to assessment 
Ccimecr results. 
CI ceo turca tes ra tre che subsystems 
The need for an integrated systems approach to per- 


sonnel management was mentioned in Section I of this paper. 


The literature supporting that need will now be reviewed. 


all 
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The basic problem is stated thus by Bray and Moses 
[zen "What is missing is a unified system that integrates 
manpower planning, selection, training, and development." 
Dneleityıns this, they continue: 


There is a heightened awareness that selection proce- 
dures do not operate im a vacuum but are part of a system. 
Ihe appropriateness of a particular seleetion standard 
depends not only on the way the job is structured, but 
upon the type and duration of training, as well as other 
Factors. A systems view of selection appears to be 
increasing in prominence. Perhaps we may one day sec a 
cohesive theory of man and woman at work. 


Os Error rai cines 15 a strong probability that 
we may move from selection systems per se to placement 
systems., To do so will require serious integration of 
existing systems, an area which is still in an embryonic 
stage. 


Miller [50] states the need in slightly different 
Boris, 


Mere cre op nc sort benavsowalliye-selewant Job Criteria, 
the comparison between relevant job behavior and an 
individual's previous work experiences, and the emphasis 
on planning an individual's work assignments in order to 
acquire a varicty of samples of his job behavior are all 
imtegral clements of a career mobility program and a 
Behaviorally-oriented identification program. Therc 
should, therefore, be an interdependence between them. 

The combination of an identification program and a career 
mobility pattern program should lead to more accurate 
selsetion decisions and consequently more effective utili- 
zation of managerial personnel. 


Thompson and Dalton [63] address performance appraisal, and 
süggest that a single system us. not adequate: 


ess to the temptation to devise one grand per- 
formance appraisal system to serve all management necds. 
Managers do have to make decisions about assignments, 
promotions, raises, and layoffs, and they also have to 
discuss performance with each man. There are different 
ooesenet mies contlicting objectives @n each of “hese 
IAS ty amo. tiem into a singde rationalized system 
May make the system less than useful for any oné purpose. 
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Management's own needs for uniformity and consistency 
should not be allowed to impose a rigid system that makes 
impractical demands on the dynamic human organization, This 
does not mean that each of these activities should be 
Nica 1solatıom setlmetmwe Contrary, the entire method 
oz handling performance, Salary increases, promotions, 
IO a5stenments Mist be Considered together. 


In explaining their reasons for Sueh a philosophy, they write: 


The appraisal system is a very poor counseling tool. 
It's possible to evaluate a man's performance, but it's 
very difficult to write your evaluation down on paper and 
Mende trend 1tto the engineer: mete toro al tie Counseling 
Session, yOU APPO Me pate how he'll respond to his 
rating three months later. If you think he will be dis- 
Satisfied and complain about it, then you only write down 
his faults. You don't tell him he's done a good job even 
if he has. In fact you don't counsel the man at all; you 
w cry tO build a case tor the rating you will later 
be giving him. 


The appraisal system favored by Thompson and Dalton is MBO. 
Fey cOMpare 1t Wie Conventional techniques as follows: 


Writers have advocated a management-by-objectives 
appraisal system for many reasons. Douglas McGregor, for 
example, favored it as a way to help a manager stop "playing 
Godn=-that 15, judging the personal worth of his fellow 
man. Alva F. Kindall and James Gatza were concerned about 
“quackery"--noting that under conventional performance 
appraisal, the manager was asked to diagnose personality 
PO them, concentratien on target=tocused appraisals 
assured a healthy emphasis on the task. Herbert H. Meyer, 
Emanuel Kay, and John R. P. French were able to show that 
Sebepetoneiledsa Me@ative Citect Gm achievement of geals, 
and that performance improved most when specific goals 
had been established. 


Slevin 159] offers a model depicting the relationship 
and contribution of the assessment center within the larger 
system: 


The assessment center seems promising as a new technique 
Foie election Ol future managers. However, the most 
laa tion of this technique will combine not 
only the appraisal function, but also the management develop- 
Mo unction of the center. The flow chart shown in 
Fegtmremi iz )eimdicates tire way in which thé assessment center 
can be used in a two pronged attack on organizational 
development. 





Figuri? 


Use of the Assessment Center as an Appraisal 
and Development Tool 
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Source: Slevin [59] 


The assessment center is conducted in usual fashion 
with an emphasis on both evaluation and the diagnosis of 
dec lopmental Meets. These wwo variables may then be fed 
WO a Ovo al aanp ower plan. This plan takes into 
ae ona a al ded on the firm and the pro- 
NOctions OL ute manpower "Weeds in various areas” Based 
on his own competence and what his developmental needs are, 
an individual will be plugged into two plans: a promotion 
Bros erh 1S Specii1ical ly constructed for the individual, 
and a management development program. The promotion pro- 
Pel eee eS a eareer Pathe that the individual is likely 
to zen ou my the organization. The management development 
program focuses on the difference between the individual's 
current competence and future needs as defined by his 
tentative career path. Using both the promotion and 
management development programs, the organization can 
implement an effective and farsighted manpower plan. 


Non EAS Lon, evidence that these concepts are not 
MESSI Ases but can be attained in practice is offered 
by Can oeil, ceal. [78]: 


Practices in this company repTesent one” of the few 
examples we found of a carcful blending of validated 
selection methods and the use of informed and systematic 
appraisal methods for administering incentives and tailor- 
Nea imeeceme lopment programs. Thus, the testing, compensa- 
tiem, awd Salary systems as well as the effective use ot 
both company and noncompany training programs arc geared 
tomara dentifying, developing, and motivating effective 
people to become effective managers. 
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No other company we visited had a program the same 
Ao co veiage even nm single areas that this cómpany's 
practices show in all the kcy areas. 


7. Unresolved Problems 
The report of a seminar conducted by the Foundation 
tor Research on Muman Bebo: [112] contains a listing of 
questions for further research. Although this list was 
aed in T957 Sathe n een Of the literature has not 
disclosed significant progress on them and they appear to 
be as timely as they@were in 1957. The listing is as follows: 


To what extent is improvement possible in the various 
iWemaeeldal Characteristics that are related to job-effec- 
tiveness? To what degree are managers born and not made? 
Are certain key attributes resistant to change? If so, 
perhaps more emphasis should be placed on selection and 
Place mnent Procedures “Ane MOst key attributes capable 
of being developed? If so, perhaps more emphasis should 
pew pliaeed on training. 


The future demand for managerial talent will not permit 
Soctmezations the luxt#iy of accepting only men with "no 
weaknesses."' In what ways can an executive be helped to 
overcome his weaknesses? What strengths can be developed 
to compensate for these weaknesses? What characteristics 
in Supervisors, peers, and subordinates can be sought to 
offset these weaknesses? What modifications can be made 
in a man's job-description or in organizational structure 
to take these weaknesses into account and yet maximize 
returns from his strengths? 


What procedures can be used in dealing with the "non- 
promotable' man, the man who is adequate in his present 
position but has reached his limit? In what ways can he 
be helped to lower his aspirations without destroying his 
motivation? 


What procedures can be weed in dealing with the sub- 
ordinates of the 'non-promotable" man? What opportunities 
cone provided for the subordinates to cContinwemtheir 
development? Should they be rotated through other units? 
Sia e ore ani zation *ssign the non-ptomotable super- 
visor activities which will take him away from his position 
frequently and permit subordinates an opportunity to under- 
study him? low may the tendency for low-grade supervisors 
te Wietract lowecrade smbordinates be dealt with? Are there 
ways in which the non-promotable man may be made more secure 
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IS pos ten So that he will be lcs» threatened*b y 
subordinates and hence be better able to develop them? 


What changes occur in an executive after he assumes a 
new managerial position? What effects do experience on the 
Job and being "in the Tole" have on his competence? Can 
these post-promotion influences be predicted? 


In studying successful performance, how much error is 
introduced by contrasting "more successful" and "less 
successful" groups of managers? Presumably, some random 
Situational factors (non-promotable boss) were responsible 
for the differences in achicvement of the two groups. More 
important, however, possibly some systematic factors 
(power tactics?) unrelated to the goals of the organization 
were responsible for some managers being promoted and 
others not. How does one take into account the effects 
Oí differential mobility in differemt @epartments? Does 
test research based @m such "success" criteria temd to 
Bespetuater errors: 


Are established principles of management development 
fully utilized? For example, are fledgling managers per- 
IT CCU propor TION on) successa Expenicnhces" 
early in their career to enable them to acquire self- 
confidence and mamwtain a high level.of aspiration? Does 
management vay enough heed to the fundamental law of 
tesining. heavyson reward, light on punishwent? What 
proportion of its management talent does an organization 
waste by using the sink-or-swim method of training managers? 


TO Mat degrees does Swmecess breed success, and failure 
Peces Ss UOeSs=success provide seli-confidence which 
enablesea man to be more imagimative, independent, original, 
and assertive? Docs failure lead to increased fear of 
failure whieh presents a man from making decisions, or 
admitting ignorance by asking questions? To what degree 
IS MS O AN ea Cire ber Scli -perpctiatime precess? 
If so, what modifications in management development, job 
a Jjenment, erc., can breaietme failure circle and rein- 
Bone ches meressreircele: 


To what extent do Managerial jobs impose rigid patterns 
of behavior even when other equally satisfactory ways of 
Xew oa ma zation l Olyectives exist? One participant 
phrased the question as follows: Is managing like fighting 
Io pay im baseball? Ihe bullfighter cannot achieve 
success hy merely killing the bull. He must pattern the 
smallest details of his performance to the rigidly held 
ects tons Of a snumber el people... Ihe baseball player, 
on tle other hand, can achieve success by maintaining a 
high batting average or by being an excellent ball hawk. 
does t Watter whether We bats with his "foot in the 
bucket," or whether his fielding is awkward. All that 
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Aa re results, Is an executive Judged successful on 
the basis of results alone or must he achieve results in 
a certain way? 


In summarizing her recent study of current assessment 
programs, Howard [34] identified future research needs as 


follors: 


A number of issues have been raised in this review that 
need to be researched, and undoubtedly there are many more. 
A brainstorming session at the Center for Creative Leader- 
pap of thesamith Rlaslardson Foundation produced a list of 
over 100 relatively unresearched questions concerning assess- 
ment centers [38]. All the components of the process and 
their integration can be studied more intensively, as can 
the potential uses of assessment procedures with such 
organizational problems as personnel selection, training, 
motivation, job analysis, job satisfaction, and organi- 
zational climate. 


Aside from the as yet unexplored research questions, 
the importance of situation related validation rescarch 
for those attempting to use assessment centers should be 
EMenusizea. because their initial industrial development 
Was groundea in research, it is often casually implied 
that therefore any assessment center will work. It is 
true that AT&T laid a solid basic foundation in its 
pioneering predictive validity research. Some other large 
companies have also contributed to the . . . (current 
knowledge of assessment center validity). 

Howard goes on to add that future program designers cannot 
ee errtorer Oud under AlcTis validity "umbrella," but rather 
must tailor both program and validation design to their 
Speci te sytbarıon. sole Further emphasizes the importance 
oie the feedback from Validation to refine the 


assessment program and keep it aligned with the demands and 


dictates of the environment. 


S MAR ot. the Literature 
faethe opinion of this writer, the greatest weakness 
of the literature on assessment centers is that it rests on 


too narrow a data base, ec.g., although there are a great 
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number of articles, the bulk of them are by a relatively few 
authors and about a small number of programs. It is speculated 
Bia AlrGgl, cohio, IBM, AMA, Sears, J. C Penney, and General 
Electric's programs together would account for an overwhelming 
MemOority OL the literature--indeed, “the first three would 
P onay CONCISO OL tne Tota Ta 

Although the reason for this is the fact that the 
Concept is still relatively new, the increase in numbers of 
programs O o se las solve tre problem, for quality 
is needed as well as quantity. This is to say that more 
research of the caliber of the Management’ Progress Study is 
Eoun ed Trswesarerto resolve Tie important problems still 
Pacing us. 


Pe econ Ol A PROTOTYPE ASSESSMENT CENTER FOR THE NAVAL 
POSTGRADUATE SCHOOL 


i nScoumpt LOns 

a. Candidates would be U. S. Naval officer students 
Piel: First quarter of the ee NManagement curriculum ef 
the Naval Postgraduate School" 

IA 220, VOUld Consist Oa mixture of U. a 
Naval officer students and faculty of the Naval Postgraduate 
school. The student assessors would be selected from students 
IM thelr crith quartêr. 

e Ucret fITexibility could beo™ebtained 
Me er ELUM to allow tarlorine of academic programs 


wo moevıcuals in accordance with assessment findings. 
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[eee laces the primary Obgective,of the assessment 
program would be research (regarding selection, placement 
and development) with a secondary objective of development. 

e. That the program would focus on the managerial 
facet of the Operational, Technical, Managerial Systems (OTMS). 

IT. “Hhat research datá collected would be privileged 
and not divulged outside of the prototype assessment center 
etait. 

o> ERI TOSoOpny 

a. In perspective, assessment centers would only be 
subsystems within the larger system of personnel management 
within the U. S. Navy. The assessment centers would, of 
necessity, interface with many other Subsystems such as 
initial selection, development, placement, selection for 
promotion, and performance appraisal. Of these (hopefully) 
complementary subsystems, performance appraisal--like assess- 
ment involves the dual roles of selection and development. 

Piasewriter mroposes that appraisal for selection 
purposes should be completely separated from appraisal used 
for feedback and/or development. Research has shown that 
selection ratings are enhanced by confidentiality. Peer and 
subordinate ratings also have potential in this area and are 
worthy of investigation. Several articles deal with this 
Sue ct including three cited earlier in Section III B he; 
Peer Ratings (e.g., Kraut [39], Amir et. a1. [2] and Hollander 
[33]). All of these supported the use of peer ratings for 


prediction of later performance. Githens and Elster [115] 
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found erbordmare ratings tommaye potential, but that their 
een was Considered thweateningemto thesmajority of Nawal officers 
surveyed. Ideally, selection appraisals should be specifically 
designed to complement the assessment center results and 
Should concentrate on those dimensions which can be best 
observed on the job. 

This writer offers management by objectives (MBO) 
as the best vehicle for the developmental aspect of performance 
appraisal. Articles on MBO abound in the literature, however 
Mebresor's classic [47] remains one of the best. He condemned 
"conventional" performance appraisal and recommended analysis 
VS. appraisal where the superior would act as a coach instead 
of a judge. | 

Deere the "ideal system. as envisioned by tias 
Writer, more than one assessment would be performed during 
Beedreci and the objæ tives of Gachewould differ. The concept 
Sorte De modcled as shown- in Figures. The assessments 
indicated in the model are explained as follows: 

PPywGdecer Entity, sseoince mo performance appraisal 
information is available (or at least none in the job being 
applied for) the hiring decision would be based solely on 
assessment data. This process would involve large numbers 
of candidates, thus, would have to be as short as possible 
while still obtaining valid results. IHE AI&T one day Parly 
Identification Program described by Moses [51] is an example 
Of this type of program. Selection would be the primary 
objective, and the brevity of the program would virtually 


Delezour any developmental aspects. 
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after four years of service, when the officer had completed 
his obligated service and elected to remain in the service 

as a career. Such a candidate would be in the rank of 
Lieutenant and, if a line officer, would have completed his 
Pinsweopenational (sea duty) tour and be commencing his first 
tour of shore duty. He would soon be faced with the decision 
IE career path to choose (Operational, Technical, 


or Managerial), and an assessment should provide information 
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not obtained in his performance appraisals which would enhance 
miema@eatlity of this decismen--to the» bemefite of the individual 
Pd tne U. S. Nany. 

(Soe lcci or nmi al Command.  —Byesthis stage 
iea career the difflerencemimecanrcer pathswmake anys norm” 
Selected a Slee E@iess Gerrcscentatevs, howewer it -sheuvdy occur 
at about 12 - 15 years of service at a rank of senior Lieu- 
tenant Commander or junior Commander. This decision ma 
crucial one for the individual as well as the organization. 
Since it represents a marked change in the degree of responsi- 
Mty and scope Sl de@msions requamed of@themofficer, an 
assessment can help combat the “Peter Principle" by probing 
those key factors of command which have not yet bcen evaluated 
on the job. 

(4) Selection for Major Command/Flag Rank or 

Early ee Onco apiu ih dopr e OI 
responsibility and scope of decisions take a quantum jump, 
therefore the benefits of assessment cited in the previous 
femme would apply here as well. For officers retiring 
carly, an assessment could provide them with an inventory 
A attributes to assist them in deciding on™a second 
Career. 

(S) Retirement,  Sallemieoricmeeratronemas early 
retirement previously described. This could be viewed as a 
M Project Transition,” and although the direct bencfits 
would acerue to the individual, the organization echrebcnerıt 


Ee irom such a “personalized” personnel management 
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pode cy , e.g., such a benefit might make a Naval career much 
Mememametractive to young officers thereby enhancing the overall 
quality Clemthe OlBieer Corps. 

It should be noted that the emphasis is on selec- 
tion initially, but there is a gradual shift toward development 
Pemmere Career progresses, until at the end, inventory and/or 
development are the primary objectives. 

Ano@rer’ pementtal@objeciive is that of placement. 
By ascertaining an officer's strengths and weaknesses with an 
able degree of accuracy, it would be possible to place 
Pim in billets better suited to his talents--as advocated by 
Pereawer 119] --thereby making better use cf the human resources 
Or the Navy. 

ces stent programs cambe classified accoldin 
to their use of signs and/or samples. "Psychometric" prograns 
Emphasize signs, while at the other end of the continuum, 
Beemavioral’ programs emphasize samples. {1 Most current programs 
fall nearer the middle of the spectrum and assess both signs 
and samples. 

d. Design of the assessment program should be based 
Ane best available information. Imgthis writer's opinion, 
mime Dest base would be data obtained from. a quality research 
effort. An alternative would be to cbtain the data from a 
Mame ltot "experts," however the validity of such data would 


be questionable. 


en À . 

Io realizes that all signs are samples, but 
whether or not a sample is a Sign is a matter determined via 
matuemtton efforts. 
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A decision would have to be made as to whether 
cer could be selected on specific dimensions unique 
let (Or fanily of billets) or whether the heterogeneity 
and number of billets would force an assessment of the "whole 
personality" on general dimensions as was done by the 0.S.S. 
[99]. This decision would properly require thorough job 
analyses which would define the billets and enable judgments 
coa aeo (it any) "ment be combined into” "job families.” 
These job analyses should be behaviorally-oriented to identify 
Significant job dimensions which can be measured by observable 
behaviors (for subsequent design of the assessment program). 

Poo ram Considerations 
amo secos 

ray. (Tie putear y Objective Ox the pro. 
Mo pe assessment center would be research focusing on the 
selection and placement of managers. Virtually all of the 
iieaeontainea on this aspect would be held in strict confi- 
dence by the assessment center staff. 

(2) Secondary. A secondary objective would be 
the individual development of participants, both candidates, 
and student assessors. The goal would be to produce a per- 
sonal development plan for each candidate which could be 
ma e anto an individual curriculum. Although the sub- 
stantive differences between individual curricula might be 
Pe eeseinee participative involvement of the student in the 
Weer mination of his academic program could produce signit1- 
cant improvement in satisfaction, motivation and, utlimately, 
performance. 
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DoC Content 
pone se h ormen ted proerem such aszth1s, a 
ere joe sigas and samples 1S considered to-be the prudent 
oce. This would offer a fuli range of instruments for 
Taon and enable the validation process to indicate which 
are most effective. Any restriction of this (i.e., using 
only signs or samples to the exclusion of the other), would 
amount to '"pre-selection” decisions without scientific basis. 
Ca cinch 
In view of the academic schedule which exists at 
the Naval Postgraduate School, this writer considers the 
five-day program to be the best length. This would roughly 
break down as three days of assessment followed by two days 
of evaluation and report writing by the assessors. 
d. Administration 
Perae eM e e edule OSES Some Challeng Tie 
scheduling problems. One solution would be to make the 
filth guarter of the management curriculum a "vertical" 
schedule for those students selected as assessors (e.g., the 
courses would be taken consecutively vice simultaneously). 
Assessommtlainingsecomld be conducted during the first peek 
of the quarter, and then assessment of twelve candidates per 
week conducted until all candidates had been assessed. Since 
iinemeemcdmdates would only be involved for three days, their 
other course work could be scheduled lightly until assessments 


were completed and then the tempo increased to compensate. 





D. DESIGN OF VALIDATION/EVALUATION/FEEDBACK SYSTEM 
These systems would differ according to the objectives 
of the program to be evaluated. Their design should be 
accomplished at the outset before the assessment program 
design is undertaken as recommended in the 0.S.S. "Assessment 
of Men" [99] as follows: 
; appraisal is the target of assessment, and it is 
ey sch "Towseeouzebauget=before taking aim. One 
should knemeahead of time, for example, whether the target 
will be an objectively determined figure, such as the 
number of units of work a man completes per day, or a 
rating of his efficiency given by competent observers 
The point that is being stressed here is that whatever 
the determinants of appraisal may be, it is important to 
distinguish them at the start, so that the assessment 
program can be designed to take account of them. 
le must be pointed out that the foregoing tacitly assumes 
the elements of appraisal selected to be relevant io ihe 
objective(s) of the assessment. 
Once developed, the appraisal system should be tested 
on a number of "successful" members of the organization, 
Mmeepoaccinle, and then revised as necessary. 
A Aal consideration is (hat the rating scales for the 


assessment and appraisal programs should be identical on 


those dimensions which are shared by both. 





Iv. JUSTIFICATION 


OOR ASSESSMENT CENTERS 

The Management Progress Study, and other AT&T studies 
which controlled for contamination, offer the most rigorous 
DADO of the concept. In addition to these, one must acknow- 
ledge the amount of resources devoted to assessment centers 
by large profit-making organizations. This is not to advocate 
gene policy 0: Tolles tie leader, "but the size of the com- 
mitments made (5 million dollar annual budget by AT&T [53]) 
are worthy of serious consideration, since no organization 
would commit such resources to a program unless they felt 
they were getting a reasonable return. AT&T has supported 
the concept for seventeen years and continued to increase its 
commitment with each passing year. The British and German 
Military programs also represent heavy commitments of long 
Standing. 
w e ABEL ISOMENT OFE A PROTOTIPE- ASSESSMENT CENTER IN 

tie: S. NAVY 

If one accepts the concept as valid, then the remaining 
atrio becomes. How to best Utilize it withan the Navy-- 
vice whether or not it should be. 

Some interesting and prophetic remarks in this regard 
were made by Rear Admiral I. E. Mode to students of the 


Urea, Navy Managenent School at Monterey [115] to wit: 





Perhaps we should take a good look at what the Army is 
doing experimentally in evaluation procedures to try and 
‘measure an officer's interest and abilities in three fields 
o combatant, administrative, and technical. This is 
being done by two full days of achievement testing and 
then validated by actual field performance tests two to 
Mmyve years later. 


This method of testing is somewhat similar to the 
Germam method used after World War Iwand used by the Bri- 
ES and 0.S.S. duapane Woeld War IT. It has considerable 
possibilities and should be carefully monitored by the 
Navy as a means of identifying truly outstanding officers 
ime the fields of operations, planning and technical branches 


Theziır Tore groups. Special contract withthe 
ies col California, iS conducting some very signif- 
icant tests in evaluation of officers at their Squadron 
Officer School (AF opposite to the Navy General Line 
School) located at Maxwell Air Force Base. This effort 
Bensists ok Student peer ratings, situational testing 
ewa ing escape andsevwasıon, ımstrucetor evaluation and 
ether psychological procedures and techniques which are 
available but not generally employed. Through monitoring 
and observing these efforts the Navy might well learn 
oc ki ronal techni dne or dental ane naval orlice Fto 
Mame OUT Giese t an the post 19/0 era. 

mre prophesy was the mention of the fields of “operations, 
etanning and technical branches which recently became reality 
as the Operational, Technical, Managerial System (OTMS). The 
wony 1s that these remarks--timely as they «sound today--were 
made on 20 February, 1959. In the ensuing years, assessment 
eenteysanne not been disproved or lost favor, but rather, as 
PO puts 1t: "a . “have reached the critical-mass 
State and ame ready to take off as more companies use then." 
This writer submits that the advent of OTMS and the All- 
Volunteer Force have made personnel decisions even more 


important than they were in the past and increase the potential 


amero the assessment center to the U. S. Navy. 
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E. INES IAE TSAMENT OF A NAVY PROTOTYPE ASSESSMENT CENTER 
AT THE NAVAL POSTGRADUATE SCHOOL (NPS) 


The Naval Postgraduate School offers a unique combination 
of candidates and assessors from a wide variety of specialtics, 
backgrounds and ranks; and necessary supporting facilities. 
These facilities include videotape capabilities as well as a 
full range of other educational media devices, and a computer 
Mentor. some other sioni icant factors are the expertise of 
the faculty (from Naval officers with operational experience 
to psychologists, psychometricians, etc.) and the liaison 
with NPS curriculum sponsors for continuous feedback on changes 
et requirements im support of the objective of a 
ment, the school offers courses in a wide variety from basic 
Skills such as reading labs, speech courses, writing, etc.; 
to more advanced work dealing with aspects of leadership, 
Interpersonal skills, organizational development, problem 
solving, decision making and human resource management. 

In considering the impact of a prototype assessment center 
upon the NPS, the following three objectives of the school 
are considered to be pertinent: 

Foster an organization and provide supporting incentives 
at NPS in order to optimize utilization of resources to 
meet the learning requirements of the students. 

Provide for continued studies in new programs, new 
PESAS teaching, better definitions of learning 
objectives and measures of effectiveness of our educational 
programs. 

Provide for increased regard, for individual diversity 


and needs on the part of students and provide an improved 
working environment for them. 
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This writer submits that significant aspects of all three 
of these objectives could be met by the proposed prototype 
assessment center. These benefits could go far toward off- 
setting the costs of the program. Also, because many of the 
costs represent sunk costs (computer center, educational 
media facilities, faculty, etc.) the actual out-of-pocket 
costs would be relatively low as compared to an industrial 
Organization which had none of these. 

In summary, the prototype assessment center could be 
established most efficiently at the Naval Postgraduate School 
MR ould benefit the school and the U. S. Navy by its 


existence. 


1.49 





Ve eee LO PEPER FORME)) 


A review OL Seccion Telde the following tasks yet 
to be performed: (The Interviews and Literature Review are 
considered complete as obres writing.) 

I. ‘Selection o- mera catar to be utilized in the evaluations. 
This would have to be done separately for each of the assess- 
Mon objectives. 

oe Job arado eso aos. (ballets). for which candidates 
would be assessed. Observable behaviors should be considered 
separately, as they would be of special interest in the design 
a the Simulations. 

3. Based on the foregoing steps, a decision must be made 
asco the Pee iene pe assessed, and further, -on the number 
of different assessment programs which would be required. If 
there 1s a great deal of heterogeneity among the jobs, a large 
number of separate assessment programs would be called for. 
AMS practical alternative to this would be a sing’ program 
Do edson.  veneral. dimensions which would attempt to Sess 
the Gestalt or whole personality as was done by the 0.S.S. [99]. 

4. Selection/design of instruments to measure the dimen- 
Sions selected (in step 5). In the prototype program consider- 
able overlap would be expected since, whenever possible, com- 
peting instruments would be included toov validation on 
both of them (e.g., provide selection on a scientific basis 


Mae or Conjecture). 
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5. Development of rating scales for use in both the assess- 
ment and evaluation programs. All dimensions common to both 
programs should have identical scales. 

6. Perform "trial runs" to test all elements of the assess- 
ment program which require testing (e.g., instruments designed 
for the program). These runs should ideally employ "successful" 
members of the organization (U. S. Navy) as test candidates. 

7. Make any revisions to the program as dictated by 
results of step 6. 

8. Develop an assessor training program. 

9. Test the assessor training program. 


REIS ea SS eS Sor training pregram as necessary, 
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VI. ANTICIPATED RESULTS 


A. SHORT TERM (Less than 2 years) 

1. An evaluation of the development objective of the 
PRES type assessment center. 

OTE iio ctiveness and efficiency of the NPS 
by enhancing the attainment of stated goals (as discussed in 
Section IV). 

te asea personal development of student assessors 
and candidates who participate in the program. 

4. Development of an in-house cxpertise in the design 


and operation of an assessment center. 


B. LONG TERM (More than 2 years) 

1. Evaluation of the selection and placement objectives 
of the prototype assessment center. 

the assessment conter proved to be effective, the 
ultimate result could be improved utilization of human resources 
within the Navy. If one accepts the premise that pcople are 
eee factor in combat, the Significance of this result 


LS obvious. 
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